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ABSTRACT 


This  dissertation  investigates  the  management  of 
systems-acquisitions  for  the  Department  of  Defense  (DOD) .  It 
indicates  how  the  DOD  can  apply  systems-engineering  technique 
for  a  more  effective  approach  to  decisions  involved  in 
developing  and  procuring  DOD  systems. 

Based  on  an  analysis  of  the  existing  policy  structure 
of  the  DOD  for  acquiring  new  systems,  a  model  is  developed 
for  the  activities  and  major-decision  points  of  that  system's 
acquisition  process  using  various  phases  and  steps  of  systems 
engineering  methodology.  The  nature  and  the  structure  of  the 
decision  process  provide  specific  areas  where  a  systems- 
engineering  methodology  could  be  applied  most  beneficially. 
After  the  criteria  are  established  for  choosing  the  appro¬ 
priate  systems-engineering  tools  and  techniques,  a  methodol¬ 
ogy  is  developed  with  specific  analysis  procedures  dependent 
on  the  stage  of  the  acquisition  process  and  the  nature  of  the 
decision  under  consideration.  A  systems-engineering  team 
with  specific  functions  and  characteristics  is  suggested  as 
a  means  by  which  the  methodology  is  implemented. 

To  support  the  conclusion  that  a  systems-engineering 
methodology  can  be  successfully  applied  to  the  decisions  in¬ 
volved  in  acquiring  DOD  systems,  a  DOD  systems-acquisition 
program  is  analyzed  in  a  historical  context.  The  program 
analyzed  is  orietnted  toward  the  development  of  a  tactical 
intelligence  system.  This  analysis  indicates  how  a  systems- 
engineering  approach  and  a  systems-engineering  team  may  be 
beneficially  applied  to  real-world  acquisition  problems. 
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EXECUTIVE  SUMMARY 


Background 

During  the  late  19  60 ' s  and  early  1970's,  systems- 
acquisition  efforts  within  the  Department  of  Defense  (DOD) 
were  beset  with  a  number  of  significant  problems.  In  1969, 
the  General  Accounting  Office  (GAO)  conducted  a  survey  of 
thirty-eight  major  veapons-systems  programs  and  found  pro¬ 
jected  costs-to-completion  15  percent  higher  than  the  origi¬ 
nal  contract  cost  figures.  In  particular  two  aircraft- 
acquisition  programs,  the  C-5A  and  the  F-lll  programs,  had 
received  a  good  deal  of  unfavorable  public  attention.  Opera 
tional  requirements  for  the  C-5A  program  were  overspecified, 
which  limited  design  trade  offs  and  led  to  significant  cost 
overruns.  The  F-lll  fighter  program  experienced  technical 
problems  and  unprecedented  cost  growth  from  approximately 
$3  million  per  unit  in  1966  to  almost  $15  million  per  unit 
in  1970.  These  problems  were  caused  by  a  variety  of  factors 
including  ineffective  program  management  on  the  government's 
part,  poor  DOD/contractor  relationships,  over  centralization 
of  systems-acquisition  management  within  the  Office  of  the 
Secretary  of  Defense  (OSD)  and  poorly  defined  operational 
requirements . 

In  1969  the  new  Republican  administration  came  into 
office  with  Melvin  Laird  as  Secretary  of  Defense  and  David 
Packard  as  his  Deputy.  3oth  Laird  and  Packard  viewed  what 
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they  found  in  the  Defense  systems-acquisition  area  with 
alarm.  It  was  clear  thnt  a  major  reorganization  of  the 
systems-acquisition  process  was  required.  Mr.  Packard,  the 
official  responsible  for  all  Defense  research,  development, 
and  procurement,  assumed  this  task. 

In  May  1969,  Packard  established  the  Defense  Systems 
Acquisition  Review  Council  (DSARC) ,  which  represented  a 
management  system  for  major  Defense  systems  acquisitions. 

The  mission  of  the  DSARC  was  "to  review  major  and  important 
Department  of  Defense  systems-acquisition  programs  at  appro¬ 
priate  milestone  points  in  their  life  cycle."  Top-level  DOD 
managers  were  to  sit  on  this  board,  review  and  evaluate  the 
programs,  deliberate  among  themselves  on  program  alterna¬ 
tives  presented  by  the  services,  and  make  recommendations  to 
the  Secretary  of  Defense  on  the  various  program  alternatives. 
In  addition  to  establishing  the  DSARC,  Packard  also  imple¬ 
mented  several  other  revisions  to  the  systems-acquisition 
process,  including  development  of  decentralized  management 
guidelines,  clarification  of  OSD  responsibilities,  adoption 
of  the  " f ly-before-you-buy"  concept,  formal  publication  of 
new  acquisition  policies  and  encouragement  of  new  contracting 
procedures . 

During  this  same  period  of  time,  procurement  policies 
of  the  overall  federal  government  had  also  come  under  close 
scrutiny.  In  the  early  197C's  the  Commission  on  Government 
Procurement  studied  these  policies  and  issued  a  report  in 
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1972  that  documented  a  number  of  recommendations  in  this 
area.  These  recommendations  have  evolved  into  a  revised  set 
of  procurement  policies  presented  in  OMB  Circular  A-109, 

Major  Systems  Acquisition ,  which  was  issued  by  the  Office  of 
Federal  Procurement  Policy  on  April  5,  1976. 

Circular  A-109  defined  a  major  system  as  "that  combi¬ 
nation  of  elements  that  will  function  together  to  produce 
the  capabilities  required  to  fulfill  a  mission  need."  Major 
programs  are  those  that  "are  directed  at  and  critical  to  ful¬ 
filling  an  agency  mission,  entail  the  allocation  of  large 
resources  and  warrant  special  management  attention."  The 
A-109  directive  was  developed  to  set  forth  policies  in  sev¬ 
eral  key  areas  as  follows:  the  strengthening  of  the  program- 
management  function,  communication  with  Congress  early  in 
the  acquisition  process,  the  designation  of  an  acquisition 
executive  for  each  agency,  the  expression  of  needs  in  mission 
terms,  the  exploration  of  alternative  systems,  and  the  reser¬ 
vation  of  four  major  program  decisions  for  the  agency  head. 

The  DOD  has  directed  the  implementation  of  the  poli¬ 
cies  of  A-109  with  two  directives,  5000.1  ( Major  Systems 
Acquisitions)  and  5000.2  ( Major  Systems  Acquisition  Process) , 
which  were  issued  in  January  1977.  These  directives  esta¬ 
blished  an  overall  DOD  svstems-acquisition-policy  framework 
and  defined  a  major  systems-acquisition  program  as  one  that 
has  a  projected  research,  development,  test,  and  evaluation 
(RDT&E)  cost  of  $75  million  or  a  projected  procurement  cost 
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of  $300  million.  In  keeping  with  the  policies  of  A-109,  the 
directives  reserved  four  key  major -program  decisions  for  the 
Defense  agency  head  (Secretary  of  Defense) : 

1.  Milestone  0  -  Decision  for  Program  Initiation 
(approval  of  need  and  advancement  to  conceptual  phase) ; 

2.  Milestone  I  -  Decision  to  Proceed  to  Demonstra¬ 
tion  and  Validation; 

3.  Milestone  II  -  Decision  to  Proceed  to  Full-Scale 
Development; 

4.  Milestone  III  -  Decision  for  Production  and 
Development. 

In  accord  with  the  policies  originally  established 
by  Deputy  Secretary  of  Defense  Packard,  the  DOD  directives 
required  the  establishment  of  the  Defense  Systems  Review 
Council  (DSARC)  with  a  membership  of  top-level-DOD  managers 
and  responsibility  to  review  major  programs  and  make  recom¬ 
mendations  to  the  Secretary  of  Defense  prior  to  major  deci¬ 
sion  milestones  I,  II,  and  III.  These  DOD  directives  also 
established  the  (Service)  Systems  Acquisition  Review  Council 
([S]SARC1  with  a  membership  of  top-level  managers  at  the  ser¬ 
vice  headquarters.  A  (S)SARC  was  established  at  each  service 
headquarters  to  review  major  programs  and  make  recommenda¬ 
tions  to  each  service  secretary  for  the  decision  process  at 
milestones  I,  II,  and  III  prior  to  consideration  of  the  pro¬ 
gram  by  the  DSARC  and  Secretary  of  Defense. 
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The  DOD  directives  also  required  that  two  separate 
documents  be  developed  for  each  major  program.  The  Mission 
Element  Need  Statement  (MENS)  is  used  to  document  the  mission 
need  and  includes  a  statement  of  the  need  in  terms  of  mission- 
element  tasks,  a  projection  of  the  enemy  threat,  an  identifi¬ 
cation  of  the  existing  DOD  capability  to  accomplish  the 
mission,  a  listing  of  constraints  for  the  problem,  and  a  pro¬ 
gram  plan  to  identify  and  explore  competitive  alternative 
systems.  The  Decision  Coordination  Paper  (DCP)  is  developed 
or  updated  prior  to  milestones  I,  II,  and  III.  It  includes 
a  number  of  program  considerations,  including  the  updated 
MENS,  acquisition  strategy,  business  plan,  management  plan, 
areas  of  program  uncertainty,  resources  required,  test-and- 
evaluation  plan,  program  issues,  DSARC  and  (S)SARC  recommen¬ 
dations,  and  Secretary  of  Defense  decisions  and  directions. 
These  documents  play  key  roles  in  the  systems-acquisition 
process  and  in  the  operations  of  the  DSARC  and  (S)SARCs. 

Analysis  of  Organizational  Goals 

The  DSARC  reflects  the  value  system  operative  within 
DOD  related  to  the  systems-acquisition  process  when  it  pur¬ 
sues  its  primary  (top-level)  goal  of  "to  acquire  a  set  of 
Defense  capabilities  which  are  adequate  to  implement  national 
policies."  Several  DSARC  subgoals  contribute  to  accom¬ 
plishing  this  top-level  goal,  including  to  be  responsive  to 
the  lead  of  Congress,  to  be  guided  by  the  directives  and 
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policies  of  the  executive  branch,  to  be  concerned  for  a 
viable  defense  industry,  to  foster  effective  systems- 
acquisition-management  capabilities,  to  be  concerned  for 
international  security  matters,  and  to  develop  the  best 
overall  program  of  systems  acquisitions.  The  (S)SARCs  have 
a  similar  set  of  goals  except  that  each  also  assumes  the 
role  of  "advocate  for  the  institutional  values  of  each  pc 
ticular  service." 

In  pursuing  the  upper  level  goal  of  developing 
best  overall  program  of  systems  acquisitions,"  the  DSA1 
the  (S)SARCs  consider  other  subgoals,  which  contribute  cs 

accomplishment.  These  subgoals  are:  to  insure  only  t  ini- 
mum  set  of  systems-acquisi tion  programs  necessary  to  ...^et 
national-defense  requirements  are  approved,  to  induce  a 
flexible  approach  to  major  systems  acquisitions,  to  minimize 
systems-development  time,  to  induce  standardization  anl 
interoperability,  to  insure  that  MENS  and  DCP  are  fully 
developed  and  properly  documented,  and  to  identify  critical 
issues  and  evaluation  factors  for  each  program.  All  of  these 
subgoals  are  important  in  the  systems-acquisition  decision 
process.  These  goals  are  presented  on  the  attached  illustration. 

Problem  Definition 

This  systems  study  addressed  the  possibility  of 
developing  a  systems-engineering  methodology  to  support  the 
systems-acquisition  decision  process.  A  problem  definition 
for  this  study  is: 
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To  determine  how  the  Defense  sy s terns -acquisition 
process  may  be  improved  using  the  techniques  of  systems 
engineering .  The  emphasis  of  this  study  is  directed  toward 
the  four  major-decision  milestones  (0,  I,  II,  and  III),  the 
operation  of  the  DSARC  and  (S)SARC  advisory  boards,  and  the 
use  of  the  Decision  Coordination  Paper  (DCP)  and  Mission 
Element  Need  Statement  (MENS)  within  the  overall  DOD-policy 
framework. 

Results  of  the  Analyses 

It  is  demonstrated  that  there  is  a  respective  pairwise 
relationship  between  phases  and  activities  of  the  systems- 
acquisition  process  and  the  phases  and  steps  of  the  systems- 
engineering  framework  (Illustrations  B  and  C) .  The  systems- 
acquisition  process  and  systems  engineering  are  also  comparable 
in  that  both  require  multidisciplinary  efforts  if  they  are 
to  be  carried  out  effectively. 

DOD  personnel  currently  involved  in  the  system- 
acquisition  process  often  do  not  view  the  process  in  the 
context  of  the  systems-engineering  framework.  Many  do  not 
see  the  overall  comparison  between  the  systems-acquisition 
process  and  a  systems-engineering  framework.  They  do  not 
perceive  the  parallelism  between  the  several  phases  of  the 
systems-acquisition  process  with  their  comparable  activities 
and  decision  points.  This  is  particularly  true  of  the 
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ILLUSTRATION  B 


THE  TIME-DIMENSION  PHASES  OF  THE  SYSTEMS -ENGINEERING 

MORPHOLOGY  COMPARED  TO  THE  PHASES  OF  THE 
SYSTEMS-ACQUISITICN  PROCESS 

Systems-Engineering 

Time-Dimension 

Phases 

Systems -Acquisition 
Life-Cyc le 

Phases 

Program  planning 

Mission  analysis  and  development 
of  MENS 

Project  planning 

Conceptual 

Demonstration  and  validation 

System  development 

Full-scale  development 

Production 

Production 

Distribution 

Deployment  or  phase  in 

Operation 

1 .  Operation/support 

2 .  Maintenance/repair 

3.  Modification/retrofit 

Retirement 

Retirement  or  phase  out 
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COMPARISON  OF  THE  LOGIC-DIMENSION  STEPS  OF  THE  SYSTEMS-ENGINEERING  MORPHOLOGY  WITH  THE 
ACTIVITIES  OF  THE  FOUR  PHASES  OF  THE  SYSTEMS-ACQUISITION  PROCESS 
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mission-analysis  (program-planning)  phase  of  the  systems- 
acquisition  process  as  compared  to  the  other  phases. 

Using  the  systems-engineering  framework,  it  was 
determined  that  the  systems-acquisition  process  is  a  stage - 
approach  problem  having  several  sequential  stages.  A 
stage  is  a  period  of  time  wherein  specific  objectives  of  a 
reasonably  well-defined  R&D  effort  are  pursued  and  that  has 
a  well-defined  decision  point  associated  with  trie  effort. 
Each  stage  of  the  process  may  be  characterized  by  four 
factors : 

1.  Clear  boundaries; 

2.  progressively  decreasing  risk  and  uncertainty; 

3.  progressively  increasing  investment; 

4.  increasing  time  required  for  completion  of  the 

stage. 

The  stage-approach  nature  of  the  systems-acquisition 
process  suggests  the  need  for  several  program-related 
criteria  for  choosing  appropriate  systems-engineering  tools 
and  techniques:  (1)  quantity  and  quality  of  available  data, 
(2)  risk  and  uncertainty  associated  with  the  program  during 
each  phase,  (3)  program  objectives,  (4)  the  systems- 
engineering  logic-dimension  step  being  analyzed,  and 
(5)  resource  requirements  of  the  program. 

Several  organizational  considerations  also  affected 
the  nature,  type,  and  degree  of  analysis  that  should  be 


included  in  a  systems-engineering  methodology  to  support 
decision  making  for  the  systems-acquisition  process: 

(1)  the  predisposition  of  an  organization  to  use  analysis 
procedures,  (2)  the  position  of  the  analysis  unit  within 
the  organizational  hierarchy,  (3)  the  background  and  experi¬ 
ence  of  the  decision  makers,  (4)  the  availability  of  the 
decision  makers,  (5)  the  mechanism  used  to  transfer  results 
of  an  analysis  to  the  decision  makers,  (6)  criticality  of 
the  decision  to  an  organization,  (7)  the  availability  of 
resources  for  the  analysis  process,  and  (8)  the  mode  of 
organizational  operation. 

Considering  the  systems-engineering  framework  and  the 
stage-apporach  nature  of  the  systems-acquisition  process, 
the  following  analysis  areas  where  the  application  of  a  systems- 
engineering  methodology  would  be  most  beneficial  are: 

The  development  of  the  Mission  Element  Need  Statement 
(the  problem  definition  in  the  program-planning  phase)  and 
the  provision  for  decision  support  at  Milestones  0,  I,  II, 
and  III.  Unified  Program  Planning  is  used  in  the  develop¬ 
ment  of  the  MENS;  justification  (viability /objectives)  check¬ 
lists,  project-scoring  models,  and  the  multiattribute  utility 
theory  are  selected  as  the  decision-support  analysis  pro¬ 
cedures  for  Milestones  0,  I,  II,  and  III,  respectively. 

The  analysis  procedures  associated  with  the  development  of 
the  MENS  and  the  provision  to  support  the  necessary  decisions 
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for  the  four  major -decision  milestones  constitute  the  systems- 
engineering  methodology  for  the  systems-acquisition  decision 
process . 

Implementation  of  Methodology 

The  implementation  of  the  methodology  would  be 
carried  out  by  the  development  and  proper  use  of  systems- 
engineering  teams  located  within  the  research  and  development 
( R&D)  staff  section  of  each  service  headquarters  and  within 
the  OSD.  These  teams  are  motivated  by  the  need  to  bridge 
the  gap  between  other  analysis  and  engineering  groups  within 
and  external  to  the  organization  and  the  various  policymakers 
and  decision  makers  of  the  organization.  In  the  case  of  the 
systems-acquisition  process,  the  policymakers  and  decision 
makers  are  represented  by  the  Secretary  of  Defense,  the 
service  secretaries,  the  DSARC  members,  and  the  (S)SARCs 
members . 

/ 

Members  of  these  multidiscipline  teams  should  possess 
graduate-level  education  in  systems  engineering,  be  mature 
and  experienced  as  analysts,  possess  operational  or  management 
experience,  be  out-going  in  their  professional  relationships, 
be  trustworthy,  and  be  innovative.  The  team  manager  should 
be  particularly  strong  in  these  characteristics.  There  are 
several  general  functions  the  teams  should  perform;  they 
should  be  cognizant  of  on-going  analysis  procedures,  monitor 
and  assist  in  the  choice  and  implementation  of  standardized 
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analysis  procedures,  advise  management  on  overall  analysis 
policies,  bilaterally  translate  information  flowing  between 
analysis  groups  and  policy/decision  makers,  and  assist  and 

advise  policy/decision  makers  relative  to  both  the  choice  of 

\ 

methodology  a^d  the  interpretation  of  results  obtained  using 
the  systems  methodology. 

The  systems-engineering  teams  would  have  specific 
functions  within  DOD  as  follows:  To  assist  DOD  in  developing 
an  overall  systems-acquisition-analysis  policy,  assist  in 
implementing  systems-acquisition-analysis  policies  and  method¬ 
ologies,  assist  in  preparing  for  DSARC  and  (S)SARC  major- 
decision  milestones,  and  assist  in  the  development  of  the 
actual  analysis  structure  for  selection  and  evaluation  of 
major-systems  programs.  The  members  of  the  systems- 
engineering  teams  would  work  closely  with  the  DOD  policy¬ 
makers  and  decision  makers  in  carrying  out  these  functions. 

Costs 

There  would  be  several  costs  associated  with  imple¬ 
menting  the  methodology.  The  primary  cost  would  be  to 
establish  an  engineering  group  of  approximately  twenty-five 
professional  personnel  within  the  OSD  and  within  the  R&D 
staff  sections  of  each  of  the  services'  headquarters.  These 
groups  would  be  divided  into  teams  of  approximately  five 
people,  each  of  which  would  be  capable  of  supporting  the 
four  or  five  systems-acquisition  programs  that  would  be 
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under  consideration  during  any  specific  period.  Each  systems- 
engineering  group  would  require  appropriate  technical  and 
administrative  support  to  carry  out  their  functions.  A 
second  cost  would  be  the  additional  management  time  required 
of  decision  makers  and  policymakers  to  effectively  implement 
the  more  rigorous  decision  process  associated  with  the 
systems-engineering  methodology.  The  final  cost  would  be 
for  the  necessary  revisions  to  policies  and  procedures  within 
DOD  associated  with  the  implementation.  However,  these 
revisions  would  not  be  significant. 

Benefi ts 

The  implementation  of  the  systems-engineering  method¬ 
ology  should  provide  several  benefits  to  DOD:  A  better 
defined,  more  highly  structured  set  of  Defense  systems- 
acquisition  problems;  a  more  equitable  evaluation  process  for 
all  programs  due  to  standardized  analysis  procedures;  the 
improved  ability  for  decision  makers  to  discriminate  between 
development  alternatives  and  whether  or  not  to  continue  the 
programs.  The  formation  and  proper  use  of  the  systems- 
engineering  teams  would  provide  decision  makers  and  policy 
makers  a  means  by  which  they  could  enhance  their  managerial 
abilities.  This  enhancement  would  be  realized  by  the  improved 
communications  between  decision  makers  and  the  various 
analysis  groups  working  on  the  programs. 
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A  historical  analysis  is  presented  for  the  Marine 
Air/Ground  Intelligence  System  (MAGIS)  acquisition  program 
for  the  period  1972  to  1975.  This  program  is  oriented  toward 
the  development  of  a  tactical  intelligence  system.  In  1972 
the  MAGIS  program  had  several  development  problems  associated 
with  it,  including  parallel  development  efforts  to  acquire 
basically  the  same  operational  capabilities,  a  lack  of 
standardization  of  hardware  and  software  design,  a  potential 
for  interoperability  problems  for  several  segments  of  the 
system,  and  the  lack  of  a  global  operational  concept  for 
the  employment  of  the  system  once  it  was  developed. 

The  need  to  develop  a  multidisciplinary  engineering 
group  to  address  the  MAGIS  acquisition  program  was  identified 
in  October  1972.  As  a  result,  during  early  1973  a  systems- 
engineering  team  was  formed  to  assist  in  developing  MAGIS. 

Over  the  next  two  and  a  half  years,  this  team  bridged  the  gap 
between  Headquarters  Marine  Corps  policy  and  decision  makers 
and  the  various  analysis  and  engineering  groups  working  on  the 
program. 

The  MAGIS  systems-engineering  team  worked  closely  with 
Headquarters  Marine  Corps'  decision  makers  on  several  key 
activities  and  decision  points,  including  the  development 
and  approval  of  an  overall  operational  concept  for  the  system; 
the  development  and  approval  of  a  new  acquisition  approach  for 
the  system,  which  eliminated  parallel  development  efforts; 
the  support  for  an  In-Progress  Review  Committee  decision  on  the 
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program;  and  the  support  for  the  Marine  Systems  Acquisition 
Review  Council,  Milestone  II  decision  for  the  system.  The 
significant  benefits  derived  from  the  employment  of  the 
systems-engineering  team  on  the  MAGIS  program  are  the  elimina¬ 
tion  of  parallel  development  efforts  with  a  projected  cost 
savings  of  over  $20  million,  the  provision  for  a  means  by 
which  decision  makers  and  policy  makers  could  more  effec¬ 
tively  carry  out  their  responsibilities,  and  the  provision 
of  a  means  by  which  the  decisions  of  Headquarters  Marine  Corps 
could  be  more  effectively  transmitted  to  the  various  analysis 
and  engineering  groups  working  on  the  program. 

Conclusions 

Based  on  the  material  developed  in  this  systems  study, 
several  conclusions  may  be  drawn.  Four  major  conclusions 
are  presented  here  in  priority  order. 

1.  The  systems-acquisition  process  may  be  effectively 
modelled  by  a  systems-engineering  framework.  The  two  are 
similar  in  that  there  is  a  respective  pairwise  correlation 
between  the  phases  and  steps  of  the  framework  and  the  phases 
and  activities  of  the  process.  Additionally,  both  the 
systems-acquisition  process  and  systems  engineering  are 
multidisciplinary  activities. 

2.  The  systems-acquisition  process  could  benefit  from 
the  application  of  a  systems-engineering  methodology  to  support 
(1)  the  problem  definition  in  the  mission-analysis  phase,  and 
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(2)  the  major-decision  milestones  of  the  process.  The 
analysis  techniques  selected  as  procedures  in  the  methodology 
should  be  dependent  on  both  organizational  considerations  and 
the  stage  of  the  development  program  under  evaluation. 

3.  Effective  implementation  of  the  methodology  is 
critically  dependent  on  the  formation  and  proper  use  of  systems- 
engineering  teams  working  with  to-level  managers  within  the 
Office  of  the  Secretary  of  Defense  and  the  several  service 
headquarters.  The  principal  role  for  these  teams  is  that  of 
bridging  the  gap  between  policy  makers  and  decision  makers  and 
the  various  analysis  and  engineering  groups  working  on  the 
program.  In  this  role,  they  can  significantly  enhance  communi¬ 
cations  for  the  development  programs  and  thereby  improve  the 
overall  management  of  the  systems-acquisition  process.  These 
improvements  would  be  reductions  in  research,  development, 

test  and  evaluation,  and  production  costs  and  in  development 
time;  they  would  enhance  supportability  and  interoperability 
once  the  system  is  deployed. 

4.  The  systems-engineering  team  associated  with  the 
Marine  Air/Ground  Intelligence  System  (MAGIS)  development 
program  (selected  for  historical  analysis)  bridged  the  gap 
between  Marine  Corps  Headquarters  (policy  and  decision  makers) 
and  the  several  analysis  groups  working  on  the  program.  The 
employment  of  the  systems-engineering  team  in  this  role  pro¬ 
vided  substantial  benefits  to  the  Marine  Corps  from  1972  to 
1975  and  is  a  good  example  of  the  successful  application  of 
the  methodology,  which  is  the  subject  of  this  dissertation. 
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Recommended.  Actions 


It  is  recommended  that  the  systems  methodology  and 
associated  analysis  procedures  developed  in  the  study  be 
applied  by  DOD  to  a  variety  of  active  systems-acquisiton 
programs.  This  recommendation  includes  the  establishment 
of  the  several  systems-engineering  teams  as  suggested  in 
the  study.  Several  other  possible  actions  that  the  federal 
government  may  want  to  pursue  are: 

1.  To  extend  the  systems-engineering-methodology 
tasks  to  other  activities  of  the  DOD  systems-acquisition 
process . 

2.  To  establish  criteria  to  determine  the  applica¬ 
bility  of  a  systems-engineering  methodology  to  the  systems- 
acquisition  process  within  other  federal  agencies. 

3.  To  develop  a  specific  systems-engineering  method¬ 
ology  to  support  the  systems-acquisition  decision  process 

in  other  federal  agencies,  such  as,  the  Department  of 
Transportation  or  the  Department  of  Energy. 

4.  To  develop  a  global  systems-engineering  method¬ 
ology  to  support  the  systems-acquisition  decision  process 
for  all  federal  agencies  from  the  perspective  of  the  Office 
of  Federal  Procurement  Policy  within  the  Office  of  Management 
and  Budget. 

5.  To  research  the  areas  of  interaction  between 
the  federal  government  and  the  industrial  sector  in  relation 
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to  the  application  of  a  systems-engineering  methodology 
to  the  systems-acquisition  decision  process. 


xxxiii 


A-  109 

ADM 

AFR 

AF  SC 

AMC 

ANMB 

AR 

ASARC 

CAIG 

CGP 

CIA 

DAE 

DCP 

DEPSECDEF 

DDR&E 

DIA 

DOD 

DS  ARC 

DT 

DT&E 

FYDP 

GAO 

GERT 

GOR 


ABBREVIATIONS  AND  ACRONYMS 

OMB  INSTRUCTIONS  ON  SYSTEMS  ACQUISITION 

ADVANCED  DEVELOPMENT  MODEL 

AIR  FORCE  REGULATION 

AIR  FORCE  SYSTEMS  COMMAND 

ARMY  MATERIAL  COMMAND 

ARMY/NAVY  MUNITIONS  BOARD 

ARMY  REGULATION 

ARMY  SYSTEMS  ACQUISITION  REVIEW  COUNCIL 
COST  ANALYSIS  IMPROVEMENT  GROUP 
COMMISSION  ON  GOVERNMENT  PROCUREMENT 
CENTRAL  INTELLIGENCE  AGENCY 
DEFENSE  ACQUISITION  EXECUTIVE 
DECISION  COORDINATING  PAPER 
DEPUTY  SECRETARY  OF  DEFENSE 

DIRECTOR  DEFENSE  RESEARCH  AND  ENGINEERING 
DEFENSE  INTELLIGENCE  AGENCY 
DEPARTMENT  OF  DEFENSE 

DEFENSE  SYSTEMS  ACQUISITION  REVIEW  COUNCIL 
DEVELOPMENT  AND  TEST 
DEVELOPMENT  TEST  AND  EVALUATION 
FIVE-YEAR  DEFENSE  PLAN 
GENERAL  ACCOUNTING  OFFICE 

GRAPHICAL  EVALUATION  AND  REVIEW  TECHNIQUE 
GENERAL  OPERATIONAL  REQUIREMENT 


xxxiv 


HQMC 
I  AC 

I  A/S  R 

II 
IOC 
IOT&E 
IP 

I  R&D 
JPL 
LCC 
MAG  IS 
MBT 
MCDEC 
MENS 
(M) SARC 
MS/OR 
MD  AC 
NMC 
NS  A 

ODCSRDA 

OFPP 

OMB 

OSM 

OPM 

OSD 

OTSE 


HEADQUARTERS,  MARINE  CORPS 
INTELLIGENCE  ANALYSIS  CENTER 

INTELLIGENCE  ANALYSIS  STORAGE  AND  RETRIEVAL 
IMAGERY  INTERPRETATION 
INITIAL  OPERATIONAL  CAPABILITY 
INITIAL  OPERATIONAL  TEST  AND  EVALUATION 
IMAGERY  PROCESSING 

INDEPENDENT  RESEARCH  AND  DEVELOPMENT 
JET  PROPULSION  LABORATORY 
LIFE  CYCLE  COSTS 

MARINE  AIR/GROUND  INTELLIGENCE  SYSTEM 
MAIN  BATTLE  TANK 

MARINE  CORPS  DEVELOPMENT  AND  EDUCATION  COMMAND 

MISSION  ELEMENT  NEED  STATEMENT 

MARINE  SYSTEMS  ACQUISITION  REVIEW  COUNCIL 

MANAGEMENT  SC I ENC E /OPE RAT I ONS  RESEARCH 

NATIONAL  DEFENSE  ADVISORY  COMMISSION 

NAVAL  MATERIAL  COMMAND 

NATIONAL  SECURITY  AGENCY 

OFFICE  OF  THE  DEPUTY  CHIEF  OF  STAFF  FOR  RESEARCH 
DEVELOPMENT  AND  ACQUISITION 

OFFICE  OF  FEDERAL  PROCUREMENT  POLICY 

OFFICE  MANAGEMENT  AND  BUDGET 

OPERATION  AND  MAINTENANCE 

OFFICE  OF  PRODUCTION  MANAGEMENT 

OFFICE  OF  THE  SECRETARY  OF  DEFENSE 

OPERATIONAL  TEST  AND  EVALUATION 


XXXV 


PERT/CPM 

POM 

PPBS 

R&D 

RDT&E 

RFP 

ROIS 

5ECDEF 

SSCNAV  INST 

SERVSEC 

S/EWCC 

SOR 

(S) SARC 
T  &E 
TERPE 

TIPI 

TRADOC 

TSOR 

VSTOL 

WPB 

WRB 

WoPAST 

5000. 1 

5000 . 2 


PROGRAM  EVALUATION  AND  REVIEW  TEC KN IOU E /CRI T I C AL 
PATH  METHOD 

PROGRAM  OBJECTIVES  MEMORANDUM 

PLANNING,  PROGRAMMING,  AND  BUDGETING  SYSTEM 
RESEARCH  AND  DEVELOPMENT 

RESEARCH,  DEVELOPMENT,  TEST  AND  EVALUATION 

REQUEST  FOR  PROPOSAL 

REMOTE  INPUT/OUTPUT  SYSTEM  • 

SECRETARY  OF  DEFENSE 

SECRETARY  OF  THE  NAVY  INSTRUCTION 
SERVICE  SECRETARY 

SIGNAL  INTELLIGENCE/ELECTRONIC  WARFARE 
COORDINATION  CENTER 

SPECIFIC  OPERATIONAL  REQUIREMENT 

(SERVICE)  SYSTEMS  ACQUISITION  REVIEW  COUNCIL 

TEST  AND  EVALUATION 

TACTICAL  ELECTRONIC  RECONNAISSANCE  PROCESSING 
AND  EVALUATION 

TACTICAL  INFORMATION  PROCESSING  AND  INTERPRETATION 
TRAINING  AMD  DOCTRINE  COMMAND 

TENTATIVE  SPECIFIC  OPERATIONAL  REQUIREMENT 
VERTICAL  AND  SHORT  TAKE  OFF  AND  LANDING 
WAR  PRODUCTION  BOARD 
WAR  RESOURCES  BOARD 

WORK  PLAN  ANALYSIS  AND  SCHEDULING  TECHNIQUE 
DOD  DIRECTIVES  ON  MAJOR  SYSTEMS  ACQUISITION 


xxxvi 


Chapter  1 


BACKGROUND  ON  THE  SYS^EMS-ACQUISITION  PROCESS 
OF  THE  DEPARTMENT  OF  DEFENSE 

Introduction 

Since  World  War  II,  research  and  development  (P&D) 
within  the  United  States  has  grown  to  be  an  industry  of 
major  proportion.  United  States  government  expenditures  for 
all  R&D  categories  have  increased  from  less  than  S78  million 
in  1940  (Perlman,  1963)  to  a  projected  total  of  $25.7  billion 
for  1979  (Mosbacker,  1979) .  Nonfederal  R&D  has  increased 
from  $2.4  billion  in  1954  (Head,  1974)  to  a  projected  total 
of  $26.1  billion  for  1979  (Mosbacker,  1979).  The  growth 
of  defense-related  R&D  has  also  been  significant,  with 
$12.7  billion  budgeted  for  fiscal  year  1979  (A.viation  Week 
Staff,  1979).  The  cost  growth  in  R&D  and  the  increasing 
complexity  of  technology  can  be  measured  by  comparing  R&D 
costs  to  overall  program  costs  for  several  systems:  R&D  was 
one  percent  of  the  total  program  costs  (R&D,  investment,  and 
operation)  for  the  F-86  Saber jet  in  the  late  1940's, 

24  percent  of  the  total  program  costs  for  the  Polaris 
Fleet  Ballistic  Missile  Program  during  the  1950 's  and  1960 's, 
and  38  percent  of  the  total  program  costs  for  Sweden's 
System  37  Vigge.n  during  the  1970's  (Head,  1974). 

In  addition  to  the  problems  of  R&D  cost  growth  and 
the  increasing  complexity  of  the  technological  base,  the 
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world  has  significantly  changed  since  1940.  First,  World 
War  II  made  it  clear  that  air,  ground,  and  sea  warfare 
cannot  be  autonomous  functions.  Rather,  these  entities  must 
be  effectively  integrated  if  success  at  modern  warfare  is 
to  be  achieved.  Secondly,  the  world  has  entered  the  atomic 
age  with  all  its  attendant  strategic  implications.  Thirdly, 
Russia  has  emerged  as  a  superpower  and  the  North  Alantic 
Treaty  Organization  (NATO)  has  evolved  to  counter  this  threat. 
All  these  changes  have  required  revisions  in  the  way  that 
national  defense  is  managed  and  that  military  systems  are 
acquired.  How  these  revisions  have  come  about  since  World 
War  II  is  the  subject  of  this  chapter. 

World  War  II 

During  World  War  II,  the  United  States  spent  more 
than  $315  billion  on  military  procurement,  production,  and 
construction.  From  early  1940  until  the  war's  end  in  1945, 
the  delivery  of  weapons,  equipment,  and  supplies  to  the 
armed  services  cost  over  $162  billion.  Many  weapons  systems 
were  developed  separately  by  the  War  Department  (Army  and 
Army  Air  Corps)  and  Navy  Department  (Navy  and  Marine  Corps) , 
including  aircraft  carriers,  battleships,  tanks,  and  various 
aircraft.  However,  the  production,  support,  and  operations 
were  coordinated  among  the  several  military  services  (Smith, 
1959)  . 
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Weapons-systems  procurement  and  the  war-planning 
function  for  the  armed  services  were  coordinated  on  a  national 
level  by  several  different  boards  and  agencies  (Birdsell, 

1976)  :  the  Army/Navy  Munitions  Board  (ANI1B)  ,  the  liar  Production 
Board  (WPB)  ,  the  Office  of  Production  Management  (OPM)  ,  the 
National  Defense  Advisory  Commission  (NDAC) ,  and  the  War 
Resources  3oard  (WRB) .  The  War  Department  had  its  own  agency 
for  the  control  and  direction  of  procurement--the  Army  Service 
Forces;  the  Navy  Department  also  had  an  agency  for  procure¬ 
ment  and  logistic  matters — the  Naval  Office  of  Procurement 
and  Material.  These  various  agencies,  which  were  necessary 
for  managing  the  overall  war  effort,  were  also  effective 
in  national  planning  and  procurement.  By  1945  it  was  clear 
that,  if  the  efforts  of  the  War  Department  and  the  Navy 
Department  were  to  be  left  totally  uncoordinated,  the  war 
would  be  much  more  difficult,  if  not  impossible,  to  win. 

The  Organization  and  Revisions  to  the 
Department  of  Defense ,  1947-1958 

The  success  of  the  national  coordination  beards 
pointed  out  the  need  for  a  civilian/defense  superagency. 

This  superagency  would  have  to  have  executive  powers  to  manage 
future  mobilization  and  to  coordinate  procurement  among  the 
several  armed  services.  In  consideration  of  these  requirements, 
the  emergence  of  the  Army  Air  Corps  as  a  de  faoto  separate 
service,  the  role  of  the  Marine  Corps  in  national  defense, 


and  other  issues.  Congress  drafted  new  legislation  estab¬ 
lishing  an  executive  coordination  authority  over  the  services 
(Halperin,  Clapp,  and  Ranter,  1974) .  This  legislation,  the 
National  Security  Act  of  1947,  established  three  separate 
departments:  the  Department  of  the  Navy  (including  the  Marine 
Corps),  the  Department  of  the  Air  Force,  and  the  Department  of 
the  Army.  The  secretaries  of  these  three  departments  were 
designated  members  of  the  President's  cabinet.  Each  depart¬ 
ment  was  a  single  entity  and  the  combined  organization  was 
the  National  Military  Establishment.  The  head  of  this 
organization,  the  Secretary  of  Defense  (SECDEF) ,  was  designated 
the  principal  assistant  to  the  President  in  all  matters 
pertaining  to  national  security  (Brady  and  Bancroft,  1973) . 

Since  1947  there  has  been  a  trend,  within  both  the 
uniformed  and  civilian  sides  of  the  Defense  organization, 
towards  greater  centraliztion .  In  1949  a  set  of  National 
Security  Act  Amendments  we re  enacted  that  downgraded  the 
Departments  of  Navy,  Army,  and  Air  Force  and  removed  the 
service  secretaries  from  the  presidential  cabinet.  However, 
the  authority  of  the  Secretary  of  Defense  was  increased  by 
making  him  the  spokesman  for  the  military  departments  in 
the  National  Security  Council.  In  1953  President 
Dwight  D.  Eisenhower,  an  advocate  of  centralization  and  unifi¬ 
cation,  appointed  Nelson  A.  Rockefeller  the  head  of  a 
committee  to  study  the  organization  of  the  Office  of  the 
Secretary  of  Defense  (OSD)  and  to  make  recommendations.  As 
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a  result  of  this  committee's  recommendations,  the  independence 
of  the  services  was  further  limited  by  making  the  chairman  of 
the  Joint  Chiefs  of  Staff  responsible  for  managing  the  Joint 
Staff,  by  eliminating  coordination  boards  with  equal  service 
representation,  and  by  tripling  the  number  of  assistant 
secretaries  from  three  to  ten.  This  last  action  had  signifi¬ 
cant  impact  on  the  services'  approach  to  military  systems 
acquisition  because  one  of  the  new  assistant  secretaries  was 
to  have  general  responsibility  for  research  and  development 
(Bauer  and  Yoshpe,  1971). 

From  1953  to  1958  concern  for  the  need  of  even  further 
revisions  grew  with  the  accelerating  pace  of  technological 
developments;  this  rapid  increase  in  technology  was  contri¬ 
buting  to  interservice  rivalries  from  the  competition  for 
scarce  Defense  dollars.  By  late  1957  the  increasing  cost 
of  weapons  systems  and  the  reactions  of  the  American  public 
to  the  technological  successes  of  the  Russians,  particularly 
in  space,  brought  significant  pressure  to  bear  for  further 
revisions  in  management  and  control.  In  early  1958  a  joint 
White  House/Pentagon  study  was  carried  out  that  resulted 
in  a  reorganization  of  the  Department  of  Defense  (DOD)  and  its  opera¬ 
tions.  These  revisions  centered  on  three  areas:  (1)  An 
increase  in  the  authority  of  the  Secretary  of  Defense 
in  planning  DOD  requirements,  administrating  the  Department, 
and  conducting  military  operations;  (2)  a  provision  for 
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a  greater  military  unification  for  planning;  and  (3)  a  pro¬ 
vision  for  unifying  military  commands  for  national  objectives 
and  policies. 

The  first  revision,  dealing  with  the  increased 
power  of  the  Secretary  of  Defense,  most  concerns  us  here. 

As  a  result  of  the  new  legislation  in  1958,  the  several 
military  departments  were  no  longer  to  be  separately  adminis¬ 
tered,  simply  separately  organized;  they  were  clearly  to  be 
under  the  direction,  authority,  and  control  of  the  Secretary 
of  Defense.  The  Secretary  could  propose  to  reassign,  consoli¬ 
date,  transfer,  or  abolish  both  tactical  and  strategic 
functions  and,  therefore,  effectively  control  the  service 
missions  and  roles;  these  modifications  could  be  limited 
only  by  Congressional  authority.  The  Secretary's  position 
was  strengthened  in  the  area  of  research  and  engineering 
by  the  establishment  of  the  position  of  the  Director  of 
Defense  Research  and  Engineering.  This  director  was  to  be 
the  Secretary's  principal  advisor  on  scientific  and  technical 
matters;  he  was  to  control  and  assign  all  research  and 
development  activities  for  the  Secretary.  This  gave  the 
Secretary  the  option  to  assign  the  development  production 
and  operational  control  of  a  system  to  whichever  service  he 
might  decide,  regardless  of  which  realized  the  need  for  the 
system.  This  option  clearly  provided  the  authority  for 
and  pointed  the  way  toward  a  centralized  approach  for 
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weapons-systems  acquisition  within  the  Department  of  Defense 
(Bauer  and  Yoshpe,  1971). 

The  McNamara  Years 

The  implementation  of  legislation  in  1958  completed 
a  series  of  major  revisions,  beginning  with  the  National 
Security  Act  of  1947,  of  the  structure  and  the  management 
practices  of  the  Department  of  Defense.  Although  the 
1958  Department  of  Defense  Reorganization  Act  placed  the 
Secretary  of  Defense  in  a  position  to  take  firm  management 
control  over  the  whole  structure,  no  secretary  chose  to  do 
so  until  Robert  S.  McNamara  came  into  office  with  the  Democra¬ 
tic  administration  in  1961.  McNamara  was  strongly  committed 
to  exercising  his  full  statutory  authority  in  managing  the 
Defense  Department  and  had  the  strong  support  of  the  President 
in  pursuing  this  goal.  As  his  Assistant  Secretary  of  Defense- 
Comptroller  ,  McNamara  chose  Dr.  Charles  J.  Hitch,  previously 
a  Rand  Corporation  economist.  Hitch  brought  with  him  and 
implemented  in  the  DOD  the  Planning,  Programming,  and  Budgeting 
System  (PPBS) ,  which  he  had  previously  developed  at  Rand.  This 
system  was  to  bridge  the  gap  between  military  planning  and 
military  budgets  and  required  that  planned  military  and  naval 
forces  be  translated  into  specifically  scheduled,  time-phased 
actions  that  could  be  equated  to  well-defined  resource 
requirements  and  budget-line  items  (Fabian,  1977). 
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The  PPBS  was  introduced  into  the  Department  of  Defense 
partially  in  response  to  systems-acquisition-related  problems. 

A  study  by  Peck  and  Scherer  (1962)  identified  six  specific 
problems  related  to  the  management  of  research,  development, 
and  production  programs  carried  out  by  industry  for  the 
Department  of  Defense:  (1)  the  schedule  slippage,  (2)  an 
increase  in  cost,  (3)  a  lack  of  qualified  government  personnel, 
(4)  the  high  frequency  of  personnel  turnover,  (5)  the  inade¬ 
quate  methods  of  cost  estimation,  and  (6)  the  insufficient 
training  in  the  measurement  and  control  of  contractor 
performance.  McNamara  introduced  the  PPBS  and  a  large 
Systems  Analysis  Group  into  the  Office  of  the  Secretary 
of  Defense  in  an  attempt  to  correct  these  problems 
and  to  more  effectively  control  the  resource-allocation 
process  within  the  Department.  The  Systems  Analysis  Group 
was  headed  by  Alain  Enthoven,  Assistant  Secretary  of 
Defense  for  Systems  Analysis,  whose  role  became  one  of  the 
most  powerful  in  DOD.  Centralized  management  at  the  depart¬ 
ment  level  became  a  reality  (Helmer,  1977). 

Secretary  McNamara  demonstrated  a  deep  and  continuing 
concern  for  weapons-systems  acquisition  during  his  tenure  in 
office.  In  a  speech  to  the  American  Society  of  Newspaper 
Editors  in  April,  1963,  he  stated  that 

...we  can  develop  only  a  fraction  of  the  systems  that  are 
proposed.  This  process  of  choice  must  begin  with  solid 
indications  that  a  proposed  system  would  really  add  something 
to  our  national  security.  The  United  States  cannot  even 
seriously  consider  going  ahead  with  a  full-scale  weapons-system 
development  until  that  basic  requirement  has  been  met. 

[McNamara,  1976] 
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The  B-70  bomber  development  program  was  cancelled  because 
in  McNamara's  opinion,  it  failed  to  meet  the  criterion  of  a 
"substantial  addition  to  national  security."  McNamara  also 
reorganized  the  operational  support  establishments  of  the 
three  military  departments.  The  Army  Material  Command  (AMC) , 
the  Naval  Material  Command  (NMC) ,  and  the  Air  Force  Systems 
Command  (AFSC)  evolved  from  this  reorganization.  Each 
command  was  to  provide  the  weapons-systems-acquisition  function 
for  its  respective  service  to  increase  the  efficiency  of  the 
procurement  and  the  support  of  new  weapons  systems  as  well  as 
to  keep  pace  with  the  rapidly  changing  technology  base 
(McNamara,  1976) . 

McNamara  further  contributed  to  the  DOD ' s  systems- 
acquisition  process  by  creating  the  Development  Concept 
Paper  (DCP,  now  referred  to  as  the  Decision  Coordination 
Paper)  to  document  the  systeras-acquisition  program.  The 
DCP  approach  was  first  formalized  in  a  Secretary  of  Defense's 
memorandum  in  September,  1967  (Defense,  1967) .  This  paper 
was  to  be  the  basic  reference  document  for  the  system, 
documenting  requirements,  plans,  projected  costs,  and  accom¬ 
plishments.  Over  time  the  DCP  has  evolved  to  the  point  where 
it  now  defines  the  development  program  and  includes  program 
objectives  and  plans,  acquisition  strategy,  areas  of  signifi¬ 
cant  risk,  performance  parameters,  development  alternatives, 
and  operational  requirements. 


The  determination  of  operational  requirements  for  the 
military  services  is  a  specific  example  of  how  the  Office  of  the 
Secretary  of  Defense  practiced  centralized  management  and 
exercised  strong  control  over  the  Department.  Assistant 
Secretary  of  Defense  for  Systems  Analysis,  Alain  Enthover, 
and  his  staff  played  a  pre-eminent  role  in  determining 
operational  requirements  and  in  defining  the  overall  problems 
of  major  weapons-systems-acquisition  programs  during  the 
McNamara  years.  This  limited  and,  in  some  cases,  pre-empted 
the  definition  by  the  separate  services  of  their  own  opera¬ 
tional  requirements  (Enthoven  and  Smith,  1971) . 

The  defining  of  requirements  at  the  OSD  level  is  an 
example  of  the  centralized  decision  making  during  the 
McNamara  years.  The  services  were  not  allowed  to  participate 
in  the  decision  process  to  the  extent  they  would  have 
desired.  The  Secretary  or  his  immediate  staff  made  most  key 
decisions  and  the  services  were  to  a  great  extent  overmanaged 
(Raymond,  1968) . 

Laird /"Packard  and  New  Approaches 
to  Systems  Acquisition 

In  1969  the  new  Republican  administration  came  into 
office  with  Melvin  Laird  as  Secretary  of  Defense. 

David  Packard  was  appointed  as  Laird's  deputy  (DEPSECDEF) ; 
an  association  began  that  was  to  have  a  great  impact  on  DOD ' s 
management  approach.  Laird  felt  Packard's  previous  experience 
with  a  major  Defense  contractor  was  a  positive  quality  and 
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enlisted  Packard's  support  in  what  became  known  as 
"participatory  management"  in  the  Pentagon  (Roherty,  1977) . 

Both  Laird  and  Packard  were  alarmed  by  what  they  found 
in  the  systems-acquisition  area  of  the  DOD.  They  were  concerned 
with  the  apparent  overcentralization  of  the  decision-making 
power  of  the  OSD  and  with  the  status  of  various  Defense 
acquisition  programs.  During  1969,  the  General  Accounting 
Office  (GAO)  conducted  a  survey  of  thirty-eight  major  weapons- 
systems  programs  and  found  that  projected  costs  to  completion 
were  50  percent  higher  than  the  original  cost  figures  on 
the  contract.  Two  aircraft-acquisition  programs,  the  C-5A 
and  the  F-lll  programs,  received  a  good  deal  of  unfavorable 
public  attention.  The  operational  requirements  for  the  C-5A 
program,  which  limited  design  trade-offs  and  led  to  signifi¬ 
cant  cost  overruns,  were  so  numerous  that  the  program  failed. 

The  F-lll  fighter  program  had  technical  problems  and  an 
unpredicted  cost  growth — from  an  estimated  $2.8  million 
per  unit  in  1962  to  an  estimated  $14.7  million  per  unit 
in  1970.  The  problem  varied  from  ineffective  program  manage¬ 
ment  on  the  government's  part  to  poor  DOD/contractor  relation¬ 
ships  and  overspecified  operational  requirements  (Helmer,  1977) . 

A  major  reorganization  of  the  systems-acquisition 
process  was  needed.  Mr.  Packard,  the  official  responsible 
for  all  DCD  research,  development,  and  production,  assumed 
this  task.  In  1970,  he  candidly  expressed  his  concern  in  a 
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speech  when  he  stated,  "Frankly,  gentlemen,  in  Defense  [DOD] 
procurement,  we  have  a  real  ness  on  our  hands,  and  the 
question  you  and  I  have  to  face  up  to  is  what  are  we  going 
to  do  to  clean  it  up"  (Helmer,  1977) .  Prior  to  this  speech, 
he  had  already  partially  answered  this  rhetorical  question  by 
issuing  two  memoranda,  one  in  May,  1969,  and  one  in  May,  1970, 
respectively  entitled  Establishment  of  a  Defense  Systems 
Acquisition  Review  Counail  and  Rolioy  Guidance  on  Major 
Weapons  Systems  Acquisition  (Defense,  1969a,  1970b). 

In  the  May,  1969,  memorandum,  Packard  established  the 
Defense  Systems  Acquisition  Review  Council  (DSARC)  ,  which  was 
intended  to  complement  the  Decision  Coordinating  Paper  (DCP, 
previously  referred  to  as  the  Development  Concept  Paper)  to 
form  a  complete  management  system  for  major  Defense  systems 
acquisitions.  This  memorandum  spelled  out  the  mission, 
functions,  composition,  authority,  and  responsibilities  of 
the  DSARC.  The  mission  of  the  DSARC  was  set  forth  as  follows: 
"to  review  major  and  important  Department  of  Defense  systems- 
acquisition  programs  at  appropriate  milestone  points  in 
their  life  cycle."  Three  major  milestones  were  identified 
as  the  review  points: 

I.  The  initiation  of  a  contract  definition, 

II.  the  transition  to  full-scale  development,  and 

III.  the  approval  of  a  system  for  production. 

Top-level  DOD  managers  were  to  sit  on  this  board  to  review 
and  evaluate  the  program  and  to  deliberate  among  themselves 
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on  recommended  program  actions  presented  by  the  several 
services.  Based  on  this  process,  the  DSARC  was  to  report  on 
the  current  status  of  the  program  and  to  make  recommendations 
to  the  SECDEF  for  the  final  decision  on  whether  to  continue 
the  program  (Defense,  1969a) . 

The  memorandum  of  May,  1970,  was  written  after 
Packard  and  his  staff  had  studied  the  systems-acquisition 
problem  for  over  a  year.  It  set  forth  new  policy  guidelines 
for  systems-acquisition  management  and  for  Defense  contracts 
and  defined  the  phases  of  the  acquisition  process  as  follows: 
conceptual  development,  full-scale  development,  and  production. 
One  principal  policy  thrust  of  the  memorandum  was  to  obtain 
and  retain  qualified  personnel  in  the  systems-acquisition 
process.  The  need  to  assign  program  managers  for  extended 
periods  to  build  up  expertise  was  also  pointed  out.  The 
development  policy  dealt  with  milestones,  risks,  trade-offs 
between  operational  requirements  and  engineering  design,  and 
questions  of  schedule.  Contract  policy  directed  that  the 
contract  be  tailored  to  the  nature  of  the  program  risks. 

In  the  memorandum,  Packard  clearly  stated  his  desire  for 
a  strong  service  participation  in  the  acquisition  of  Defense 
systems.  He  pointed  out  the  need  for  OSD  to  establish  acqui¬ 
sition  policies  and  to  evaluate  the  performance  of  the  services 
which  were  to  develop  and  procure  the  major  weapons  systems. 
(Defense ,  1970b). 


During  this  same  time,  the  DOD ' s  management  practices 
were  undergoing  a  formal  analysis  by  the  Blue  Ribbon  Defense 
Panel.  This  panel's  staff  director,  J.  Fred  Buzhart, 
was  later  the  General  Counsel  of  the  DOD.  Under  his  direc¬ 
tion  the  panel  developed  a  final  report  (Defense,  1970a)  that 
outlined  a  set  of  recommendations  relating  to  procurement  and 
contracting  procedures.  This  report  was  given  to  the 
President  and  the  SECDEF  on  July  1,  1970.  Both  Laird  and 
Packard  found  some  of  the  panel's  recommendations  restric¬ 
tive  but  the  report  did  affect  the  overall  revision  of  the 
systems-acquisition  process. 

Packard's  two  memoranda  and  certain  recommendations 
from  the  Blue  Ribbon  Defense  Panel  formed  the  basis  for  a 
new  systems-acquisition  directive;  DOD  Directive  5000.1 
was  issued  on  July  13,  1971  (Defense,  1977a).  This  directive 
restated  the  policies  previously  established  and  went  further 
in  delineating  OSD  and  service  responsibilities.  The  directive 
established  several  criteria  for  deciding  whether  a  system 
could  be  considered  a  major  program:  (1)  If  estimated  research, 
development,  test,  and  evaluation  (RDT&E)  costs  were  over 
$50  million  or  production  costs  were  over  $200  million;  (2)  if 
there  were  a  national  emergency,  and  (3)  if  there  were  a  recommenda¬ 
tion  from  DOD  component  heads  or  OSD  officials.  The  order 
covered  program  responsibilities;  the  conduct  of  a  program, 
including  milestones/DCP  development;  and  the  DSARC  require¬ 
ments.  It  also  covered  the  definition  of  requirements,  cost 


parameters,  logistical  support,  uncertainties,  test  and 
evaluation,  contacts,  and  source  selection.  The  order  was 
to  apply  to  all  military  departments  and  to  other  DOD  agencies, 
such  as,  the  Defense  Communications  Agency. 

Another  innovation  of  Packard's  was  to  introduce 
the  concept  of  prototyping  into  Defense  systems  acquisition. 
Although  this  fly-before-buy  approach  was  not  new  (it  had 
been  used  for  aircraft  development  prior  to  World  War  II) , 
this  was  the  first  time  it  had  been  established  as  a  formal 
policy  within  the  Department  of  Defense.  In  its  most 
desirable  form,  prototyping  meant  that  competing  contractors 
produced  a  limited  number  of  preproduction  or  prototype 
end-items  for  test  and  evaluation  prior  to  the  award  of 
the  production  contract  (Roherty,  1977) .  This  approach 
could  not  be  applied  to  all  acquisition  programs;  but  the 
F-16  program,  where  General  Dynamics  and  Northrop  competed 
during  development,  is  an  excellent  example  of  the  implemen¬ 
tation  of  prototyping.  Packard  (1973)  outlined  the  benefits 
of  prototyping  to  include  improvements  in  total-program-cost 
estimates,  in  the  means  by  which  the  operation  of  a  system 
could  be  evaluated  before  production,  in  the  verification 
of  new  technology,  and  in  the  enhancement  of  a  systems 
reliability.  Massey  (1973)  picked  up  on  Packard's  theme 
and  suggested  that  already  operational  systems  could  be 
improved  by  using  an  experimental  prototyping  approach  to 
operational  RDT&E . 


Packard  was  also  deeply  concerned  that  the  services, 
particularly  the  program  managers,  should  manage  their  own 
programs.  He  felt  that  the  services  had  the  primary  respon¬ 
sibility  for  acquiring  new  systems.  He  stated,  "My  study 
convinces  me  that  during  the  past  few  years,  as  programs 
got  into  trouble,  OSD  offices  became  too  deeply  involved  in 
second  guessing  the  services  and  in  making  overriding 
decisions"  (Packard,  1971).  He  insured  that  the  services 
did  their  own  concept  definition  for  new  systems;  the  DCP ' s 
were  to  originate  with  the  service  having  the  requirement 
and  not  with  the  Office  of  the  Director  of  Defense  for 
Research  and  Engineering  ( DDR&E) . 

Packard's  interest  in  service  prerogatives  is 
reflected  in  the  DSARC's  record  for  1970.  During  that  year 
twenty-one  DSARCs  were  held;  eleven  times  the  DSARC  approved 
a  service-recommended  action;  six  times  it  granted  limited 
approval;  three  times  it  requested  the  service  go  back  and  do 
further  work;  and  one  time  it  deferred  a  decision  pending 
Congressional  support  (Packard,  1971) . 

Barry  J.  Shillito  (1971),  then  Assistant  Secretary 
of  Defense,  stressed  the  following  revisions  to  systems-accuisition 
management  during  DEPSECDEF  Packard's  tenure  in  office: 
the  establishment  of  the  DSARC,  the  development  of  decentral¬ 
ized  guidelines,  the  clarification  of  OSD  responsibilities, 
the  establishment  of  milestones,  the  adoption  of  a 
fly-before-you-buy  concept,  the  formal  publication  of 
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new  acquisition  policies,  and  the  encouragement  of  new 
contracting  procedures.  Additionally,  service  involvement 
in  the  acquisition  process  was  emphasized,  with  particular 
concern  for  the  prerogatives  of  the  program  .anagers. 

These  new  policies  and  procedures  for  s;s*-.e~  -guisition 
contrasted  markedly  with  the  approach  during  'he  Mclanara 
years;  they  set  the  framework  and  tone  foi.  systems  acquisi¬ 
tion  in  the  1970’s. 

The  Impact  of  the  Office  of  Management 
and  Budget's  Policies 

During  the  late  1960’s  and  early  1970’s,  Congress  was 
also  concerned  about  federal  procurement  policies.  As  a 
result  in  November,  1969,  Congressional  legislation  was  passed 
that  created  the  Commission  on  Government  Procurement.  This 
commission  was  to  study  federal  procurement  policies  and 
procedures  and  to  make  recommendations  to  Congress  concerning 
the  promotion  of  practical,  cost-effective  and  -efficient 
means  of  procurement  by  the  federal  government's  Executive 
Branch.  The  work  of  this  commission  resulted  in  the  Report 
of  the  Commission  on  Government  Procurement,  which  was  given 
to  Congress  and  the  Executive  Branch  in  December,  1972. 

This  report  contained  recommendations  for  establishing  the 
need  and  objectives  of  new  programs  related  to  the  basic 
missions  of  the  government  agencies.  It  also  suggested  that 
alternative  development  approaches  and  the  means 
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to  select  preferred  alternatives  be  explored  and  system- 
implementation  questions  be  answered. 

The  commission's  recommendations  had  a  significant 
impact  on  the  federal  government's  perception  of  how  to 
carry  out  procurement.  The  executive  branch's  consideration 
of  the  commission's  recommendations  resulted  in  the  Office 
of  Management  and  Budget's  (OMB)  issuing  0MB  Circular 
No.  A-109  on  April  5,  1976,  entitled  Major  Systems  Acquisi¬ 
tions.  This  circular  established  policies  to  be  followed 
by  executive-branch  agencies  when  acquiring  major  systems  and, 
therefore,  applied  to  the  acquisition  of  weapons  systems  by 
the  DOD. 

OMB  Circular  No.  A-109  pointed  out  that  the  manage¬ 
ment  of  the  acquisition  of  major  systems  should  include 
analyzing  agency  missions;  determining  mission  needs;  setting 
program  objectives;  determining  system  requirements;  planning, 
budgeting,  and  funding  the  program;  RDT&E ;  contracting; 
producing  the  system;  establishing  program  and  management 
controls;  and  introducing  the  system  into  use.  It  also 
outlined  a  set  of  definitions  for  terms,  including  agency 
mission,  mission  need,  program  objectives,  program,  major 
system,  life-cycle  cost,  and  sy stems -acquisition  process 
(Appendix  A) . 

The  circular  also  established  the  following  general 
policies  for  federal  agencies  acquiring  major  systems; 
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1.  Express  needs  and  program  objectives  in  mission 
terms,  not  equipment  terms. 

2.  Emphasize  the  early  and  continuous  activities 
in  the  systems-acquisition  process  that  will  stimulate 

a  competitive  exploration  of  alternative  systems-design 
concepts  in  response  to  mission  needs. 

3.  Convey  to  Congress  the  plans  for  each  major 
systems  acquisition  relating  the  system  to  agency  mission 
needs. 

4.  Establish  clear  lines  of  authority  and  responsi¬ 
bility  for  systems  acquisition  and  obtain  agency-head  approval 
at  key  decision  points  in  the  acquisition  process. 

5.  Designate  an  acquisition  executive  for  each 
agency  to  integrate  and  unify  the  management  process  for 
the  agency's  major  systems  acquisitions. 

6.  Establish  a  program-management  office  for  each 
major  systems  acquisition. 

7.  Rely  on  private  industry  for  support  of  the 
systems-acquisition  process  in  accordance  with  the  policies 
outlined  in  OMB  Circular  No.  A-76  (0MB,  1977)  ,  which  required  the 
minimization  of  in-house  government  work  that  could  be 
accomplished  in  the  private  sector. 

Major  systems-acquisition  management  objectives  were 
also  set  forth  in  the  circular  and  are  briefly  summarized  below 

1.  To  insure  that  each  major  system  fulfills  a 
mission  need; 


2.  to  provide  for  competition  among  various  systems- 
design  concepts  throughout  the  entire  acquisition  process; 

3.  to  insure  that  there  are  appropriate  trade-offs 
among  investment  costs,  ownership  costs,  schedules,  and 
performance  characteristics; 

4.  to  establish  a  strong  set  of  checks  and  balances 
by  adequate  test  and  evaluation  of  the  system; 

5.  to  plan  the  acquisition  of  a  system  based  on  an 
analysis  of  the  agency  mission,  including  an  appropriate 
resource-allocation  process  resulting  from  a  clear  statement 
of  agency-mission  needs; 

6.  to  tailor  an  acquisition  strategy  to  each  program, 
refining  the  strategy  as  the  program  proceeds  through  the 
acquisition  process. 

The  circular  also  stated  that  technical  and  program 
decisions  should  normally  be  made  at  the  level  of  the  agency 
or  operating  activity.  However,  the  circular  specified 
that  the  following  four  key  decisions  should  be  made  by  the 
agency  head: 

1.  The  identification  and  definition  of  the  specific 
mission  need  to  be  fulfilled,  the  relative  priority  assigned 
within  the  agency,  and  the  general  magnitude  of  resources 
that  could  be  invested; 

2.  the  selection  of  competitive  systems-design 
concepts  to  be  tested  or  demonstrated  or  the  authorization 
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to  proceed  with  the  development  of  a  noncompetitive 
(single-concept)  system; 

3.  the  commitment  for  the  full-scale  development 
and  limited  production  of  a  system; 

4.  the  commitment  for  the  full  production  of  a  system. 

The  requirements  of  the  agencies  to  prepare  for  the 

initial  milestone  through  mission-need  analysis  became  a 
key  element  of  the  policies  set  forth  in  the  circular. 

Mission  needs  had  to  be  determined  from  an  analysis  of  the 
agency's  mission  reconciled  with  the  overall  capabilities, 
priorities,  and  resources.  The  circular  further  stated  that 
a  need  should  not  be  defined  in  equipment  terms  but  should 
be  defined  in  terms  of  the  mission,  the  purpose,  the  capa¬ 
bility,  the  agency  components  involved,  the  schedule  and 
cost  objectives,  and  the  operating  constraints.  A  mission 
need  was  to  be  established  independently  of  any  particular 
system  or  technological  solution  and  was  to  be  identified, 
well  defined,  and  documented  prior  to  the  initial  decision 
point  of  the  system-acquisition  process. 

The  overall  framework  for  the  systems-acquisition 
process  as  established  by  the  policies  of  A-109  (OMB,  1976) 
is  graphically  portrayed  in  Figure  1.  An  analysis  of  mission 
needs  is  on  the  left  of  the  chart,  with  consideration  for 
overall  capabilities,  priorities,  and  resources.  The  process 
moves  to  the  right  as  a  function  of  time  and  passes  through 
the  four  major  decision  milestones  for  the  validation  of  a 


Figure  1.  Sequence  of  Activities  and  Decisions  to  Implement  the 
Policies  of  OMB  A-109  (1976). 
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mission  need,  the  selection  of  concepts  for  demonstration,  the 
selection  of  the  alternatives  for  full-scale  development, 
and  the  production  decision.  Each  decision  is  subject  to 
agency-head  approval. 

OMB  Circular  A-109  (1976)  was  reasonably  well  received 
within  the  federal  establishment  and  the  private  sector. 
Ilusgrave  (1976)  ,  writing  from  the  perspective  of  a  naval 
officer  involved  in  DOD  systems  acquisition,  provided 
a  favorable  review  of  the  policies  of  A-109  and  the  initial 
implementation  steps.  Some  benefits  he  attributed  to  the 
A-109  approach  were  the  improved  linkage  between  resource 
inputs  and  service-program  needs;  a  high  incidence  of  aggres¬ 
sive  and  innovative  problem  solving,  the  integration  of 
technical  disciplines  into  productive-  system-oriented  teams, 
and  the  avoidance  of  suboptimal  designs  by  looking  at  the 
total  problem,  not  a  component  of  it. 

The  importance  of  A-109  in  planning  for  major  Defense 
systems  and  its  impact  on  federal  procurement  was 
outlined  by  Coleman  (1978) .  He  pointed  out  that  the  Navy 
was  criticized  by  GAO  for  requesting  information  from 
industry  on  vertical  take-of f-and-landing  aircraft  sub¬ 
systems — engine,  airframe,  and  avionics — instead  of  for  the 
system  as  a  whole,  as  A-109  required.  Harlamor  (1973) 
indicated  general  aerospace-industry  support  for  A-109  but 
felt  careful  implementation  would  be  required  for  the  direc¬ 
tive  to  be  successful.  He  also  noted  the  unqualified 
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support  of  the  aerospace  industry  for  the  portion  of  A-109 
that  referred  to  OMB  Circular  A-76  (1977)  and  its  requirement 
to  use  industrial  sources  in  preference  to  government  in-house 
capabilities  to  carry  out  development  efforts. 

An  Overview  of  the  Current  Process 

Since  A-109  was  issued  in  April,  1976,  the  Depart¬ 
ment  of  Defense  has  taken  significant  steps  to  implement  the 
policies  in  the  circular.  A  revised  version  of  DOD 
Directive  5000.1,  Mac  or  Systems  Acquisition,  was  issued  on 
January  18,  1977.  This  was  an  update  of  the  previous  direc¬ 
tive  of  that  number,  which  could  be  traced  back  to  the 
original  directive  of  July,  1971,  and  DEPSECDEF  Packard's 
1969  memorandum.  A  complementary  order,  DOD  Directive  5000.2, 
Major  Systems  Acquisition  Process  ,  was  also  published 
on  January  13,  1977.  These  two  directives,  along  with  DCD 
Directive  5000.3,  Test  and  Evaluation  of  April,  1978, 
effectively  established  the  acquisition  policies  of  A-109 
for  the  DOD  (Defense,  1977a,  1977b,  1978). 

The  provisions  of  the  new  DOD  Directive  5000.1 
defined  major  systems  as  those  that  were  so  designated  by 
the  SECDEF ,  those  that  were  recommended  by  DOD  component  heads 
and  OSD  officials,  and  those  with  an  anticipated  cost  of 
S75  million  for  RDT&E  or  $300  million  to  produce.  The 
following  four  key  SECDEF  decisions  were  to  be  made  at 
distinct  phases  of  the  program: 
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1.  Milestone  0  -  the  decision  to  initiate  a  program; 

2.  Milestone  I  -  the  decision  to  proceed  to  demon¬ 
stration  and  validation; 

3.  Milestone  II  -  the  decision  to  proceed  to  full- 
scale  engineering  development; 

4.  Milestone  III  -  the  decision  to  produce  and 

deploy. 

The  overall  acquisition  policies  set  forth  in  5000.1 
(Defense,  1977a)  were  consistent  with  those  already  mentioned 
for  A-109  (CMB,  1976).  In  fact  the  new  5000.1  was  generally 
consistent  with  previous  versions  of  the  directive  with  one 
major  exception.  Although  the  definition  of  a  problem  and  the 
identification  of  requirements  had  been  major  considerations 
for  years  with  systems-acquisition  programs,  this  was  the  first 
time  that  a  SECDEF  decision  was  mandatory  to  approve  the 
requirements  and  intitate  the  program. 

The  5000.1  directive  stated  that  before  a  Milestone  0 
decision  point  could  be  reached,  the  SECDEF  must  have  requested 
a  new  capability,  or  a  DCD  component  (one  of  the  services) 
head  must  have  realized  a  mission  need  and  determined  that 
a  new  capability  was  necessary  to  meet  that  need.  Once  this 
point  was  reached,  the  DOD  component  head  must  submit 
a  statement  of  the  mission  need  to  the  SECDEF  and  request 
approval  to  identify  and  explore  alternative  solutions.  The 
directive  also  stated  that  support  for  determining  the 
mission  need  must  have  been  documented  in  the  Mission  Element 
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Need  Statement  (MENS) .  Cnee  a  mission  need  was  determined 
to  be  essential  and  reconciled  with  other  DOD  capabilities, 
resources,  and  priorities,  the  SECDEF  would  approve  the  mission 
need.  After  this  decision,  one  or  more  of  the  DOD  components 
must  systematically  and  progressively  explore  and  develop 
alternative  systems  concept. 

The  DOD  Directive  5000.2  (Defense,  1977b)  amplified 
the  general  policies  of  5000.1  (Defense,  1977a)  and  provided 
definitive  guidelines  for  advisory  councils,  program  reviews, 
mission-area  analyses,  program-management  considerations, 
and  program  documentation.  Two  advisory  councils  were  estab¬ 
lished  by  the  5000.2  directive  to  review  major-systems  programs 
at  Milestones  I,  II,  and  III  and  to  make  recommendations  to 
the  SECDEF.  The  DSARC,  the  senior  council  established  by 
the  order,  had  been  used  by  DOD  since  DEPSECDEF  Packard's 
1969  memorandum  (Defense,  1969a);  the  DSARC  charter  outlined 
its  membership,  its  participants  and  advisors,  and  its  opera¬ 
tions.  The  procedures  to  be  used  by  the  DSARC  were  generally 
consistent  with  those  of  previous  years.  A  Defense  Acquisition 
Executive  (responsibilities  outlined  in  DOD  Directive  5000.30 
(Defense,  1976])  was  designated  as  the  Chairman  of  the  DSARC. 

A  graphical  representation  of  the  number  of  DSARCs  held  since 
DEPSECDEF  Packard  initially  directed  this  management  approach 
is  provided  in  Figure  2. 

The  second  advisory  council  established  by  5000.2 
was  the  (Service)  System  Acquisition  Review  Council  ((S]SARC). 
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This  directive  required  that  each  Service  Secretary  charter 
a  (S)SARC  similar  in  composition,  responsibilities,  and 
operation  to  the  DSARC.  The  (S)SARC  was  to  review  major 
systems-acquisition  programs  and  to  provide  advice  to  the 
Service  Secretary  either  supporting  a  subsequent  DSARC  or, 
for  selected  programs,  making  recommendations  to  the  SECDEF. 
Either  the  Service  Secretary  or  the  Under  Secretary  must 
chair  the  (S)SARC.  DSARCs  and  (S)SARCs  must  be  conducted 
at  each  milestone  decision  point  for  all  major  systems- 
acquisition  programs  except  when  specifically  waived  by  the 
SECDEF . 

The  5000.2  directive  provided  the  authority  for 
the  SECDEF,  with  the  assistance  of  the  DOD  components,  to 
establish  mission  areas  that  reflected  several  operating 
categories  essential  to  the  accomplishment  of  a  Defense 
mission.  The  DOD  component  heads  were  directed  to  carry 
out  a  continuing  analysis  of  their  assigned  mission  respon¬ 
sibilities  in  the  several  mission  areas  in  order  to  identify 
those  mission  elements  for  which  the  existing  or  projected 
capability  was  deficient  and  to  identify  opportunities 
for  enhancing  the  existing  capabilities. 

The  5000.2  directive  also  covered  program-management 
considerations  relating  to  the  program  manager's  responsi¬ 
bilities  and  his  position  in  the  overall  DOD  management 
structure,  the  developmental  status  of  subsystems  projected 
for  use  in  the  overall  system,  and  the  importance  of  the  MENS 
and  its  relationship  to  the  Five-Year  Defense  Plan  (FYDP) , 
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Program  Objectives  Memorandum  (POM),  and  the  PPBS .  The 
policies  related  to  management  in  5000.2  also  included 
acquisition  strategies,  costs  of  acquisition  and  ownership, 
competitive  demonstrations,  contract  considerations,  and 
management  information  systems. 

Two  key  documents,  the  MENS  and  the  DCP,  were 
required  by  5000.2  to  support  the  DSARC  and  (S)SARC  reviews 
and  to  aid  the  SECDEF  in  making  the  best  decision.  The 
Mission  Element  Need  Statement  required  that  the  ser¬ 
vice  with  the  need: 

1.  Identify  the  mission  area  and  state  the  need  in 
terms  of  the  task  to  be  performed; 

2.  assess  the  projected  enemy  threat;. 

3.  identify  the  existing  DOD  capability; 

4.  assess  the  deficiency  in  the  existing  capability; 

5.  state  the  known  constraints  applicable  to  any 
acceptable  solution; 

6.  assess  the  impact  of  not  acquiring  the  capability; 

7.  provide  a  program  plan  to  identify  and  explore 
competitive  alternate  systems. 

The  Decision  Coordinating  Paper  (DCP)  required  that 
MENS  be  completed,  updated,  and  incorporated  into  the  DCP  and 
the  following  be  provided: 

1.  The  alternative  programs; 

2.  the  acquisition  strategy; 

3.  the  short-  and  long-term  business  planning; 
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4.  the  program  structure  and  management  plan; 

5.  the  program  uncertainties; 

6.  a  technology-assessment  annex,  including  areas 
of  technological  risk  remaining  in  the  program; 

7.  a  resource-input  annex,  including  cost,  produc¬ 
tion,  and  inventory  data; 

8.  a  logistics-support  annex; 

9.  the  program  schedule,  cost,  and  performance 
information  for  Milestones  II  and  III; 

10.  the  test  and  evaluation  planning  and  status; 

11.  the  DSARC  and  (S)SARC  results  and  recommendations; 

12.  the  SECDEF  decisions. 

The  5000.2  directive  outlined  the  requirements  for 
processing  and  coordinating  the  DCP.  The  directive  identified 
three  separate  sets  of  program  issues  that  must  be  considered 
by  the  DSARCs  and  (S)SARCs  at  each  milestone.  Past  DSARC 
and  (S)SARC  actions  in  relationship  to  the  DCP  were  also 
covered  in  5000.2. 

An  overview  of  the  systems-acquisition  process  from 
the  OSD-decision  viewpoint  is  graphically  depicted  in  Figure  3 
The  process  begins  at  the  top  of  the  DELTA  chart,  with 
a  continuing  mission  analysis  and  the  existing  technical 
base.  A  needed  capability  is  then  identified  and  translated 
into  a  MENS.  Once  the  MENS  is  approved,  phases  of  the  acqui¬ 
sition  program  alternate  with  DSARC/DCP  decision  points.  The 
decision  points  always  proceed  a  major  phase  of  the  acquisition 


program.  These  decision  points  provide  a  major  review  of 
the  program  status  before  approval  is  given  to  proceed  to 
the  next  milestone.  The  final  major-decision  point  is 
Milestone  III,  the  production/deployment  decision,  which 
leads  to  a  new  operational  capability. 
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Chapter  2 


THE  RESEARCH  PROBLEM:  ITS  NATURE,  PURPOSE,  AND  SCOPE 

Introduction 

Major  revisions  of  the  weapons-systems  acquisitions 
took  place  within  the  Department  of  Defense  (DOD)  in  the 
early  1970 's.  The  Deputy  Secretary  of  Defense  (DEPSECDEF) 

David  Packard  and  others  on  the  Office  of  the  Secretary 
of  Defense  (OSD)  staff  instituted  a  new  set  of  acquisition 
policies  that  included  the  establishment  of  the  Defense 
Systems  Acquisition  Review  Council  (DSARC) ,  the  introduction 
of  a  fly-before-you-buy  concept,  the  promotion  of  proto¬ 
typing  or  dual  development,  and  the  formal1  publishing  of 
the  systems-acquisition  milestones  for  management  procedures. 
These  innovations  were  quickly  followed  by  the  promulgation 
of  Circular  A-109  by  the  Office  of  Management  and  Budget  (OMB) 
in  1976.  This  circular  established  new  systems-acquisition 
policies  for  all  executive  agencies  and  had  particular  impact 
on  the  DCD ' s  approach  to  the  requirements-analysis  portion 
of  the  systems-acquisition  process.  The  circular  also  directed 
that  the  mission-area  analysis  continue  and  required  that  a 
Mission  Element  Need  Statement  (MENS)  be  developed  for  each 
major  program.  Two  revised  DOD  directives,  5000.1  {Major 
Systems  Acquisitions )  and  5000.2  {Major  Systems- Acquisition 
Process)  were  issued  in  1977  to  implement  the  new  policies 
in  OMB  Circular  A-109  (Defense,  1977a,  b;  OMB,  1976). 
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Circular  A-109  and  the  two  DOD  directives  established 
a  strong,  comprehensive,  and  well-conceived  policy  framework 
within  which  to  manage  DOD  systems  acquisitions.  In  light 
of  this  policy  framework,  the  process  of  acquiring  Defense 
systems  should  have  proceeded  smoothly,  with  a  minimum  of 
problems.  However,  this  was  not  the  case;  wide-ranging 
problems  have  been  identified  in  the  systems-acquisition 
process  during  the  1970's.  Some  problems  of  the  1970 's 
were  holdovers  from  a  previous  time,  such  as,  the  Main  Battle 
Tank-70  (MBT-70)  Program  (Croskery  and  Horton,  1974) . 

Other  problems,  such  as,  our  growing  dependence  on  the  foreign 
production  of  critical  components  and  the  minimizing  of 
Defense  procurement  since  the  Vietman  War  (Gansler,  1977) , 
are  beyond  the  scope  of  the  systems-acquisition-policy 
framework  discussed  above.  But  many  problems  that  arose 
within  Defense  research  and  development  (R&D)  and  the  systems- 
acquisition  process  do  fall  within  the  scope  of  the  policy 
framework:  system  testing  and  evaluation  (Harrison,  1976) ; 
requirements  analysis  (Jordan,  1974) ;  software  development 
and  standardization  (DeRoze,  1975) ;  and  system  reliability, 
maintainability,  and  availability  (Gansler,  1976) . 

Many  R&D-related  problems  within  the  DOD  were 
caused  by  the  size  and  complexity  of  the  procurement  programs 
for  such  diverse  systems  as  cruise  missiles  (Canfield  and 
Kellett,  1973);  guided-missile  frigates  (Beecher  and  DiTrapani, 
1973);  command,  communications,  and  control  systems  (Berry, 
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1978) ;  and  vertical-and-short-take-of f-and-landing  (VSTOL) 
aircraft  (Coleman,  1978) .  The  diversity  of  these  acquisition 
programs,  coupled  with  the  variety  of  development  problems 
similar  to  those  mentioned  above,  have  continued  to  challenge 
the  DOD  management  structure.  Therefore,  the  planning, 
control,  and  management  of  the  systems-acquisition  process 
is  still  a  major  concern  within  the  DOD. 

One  cannot  speak  of  the  optimization  (in  the  mathema¬ 
tical  sense)  of  the  systems-acquisition  process  because  of 
the  broadness  and  complexity  of  the  problem.  However, 
the  improvement  of  the  systems-acquisition  process,  particu¬ 
larly  relative  to  the  four  major-decision  milestones  (0,  I, 
II,  and  III),  is  certainly  a  meaningful  goal — a  goal  which 
has  been  pursued  in  this  dissertation  by  using  systems 
engineering  while  considering  the  DOD  policy  framework. 

The  research  problem  associated  with  this  goal  is  discussed 
here;  the  purpose  and  objectives  of  the  research  are  also 
presented.  Comments  are  made  on  the  scope  and  focus  of 
this  study  and  on  the  related  literature;  the  overall 
organization  of  the  report  is  also  outlined. 

The  Problem  Statement  and  Its  Nature 

Keeping  in  mind  the  description  of  the  systems- 
acquisition  process  in  the  first  chapter  and  the  comments 
made  in  the  above  introduction,  the  overall  research  problem 
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is  to  determine  how  the  Department  of  Defense  systems- 
aoquisition  process  may  be  improved  using  systems  engineering. 

The  emphasis  of  this  study  will  be  directed  toward  the  four 
major-decision  milestones  (0,  I,  II,  and  III),  the  operation 
of  the  Defense  Systems  Acquisition  Review  Council  and 
the  (Service)  Systems  Acquisition  Review  Council  (tS)SARC) 
advisory  boards,  and  the  use  of  the  Decision  Coordination 
Paper  (DCP)  and  Mission  Element  Need  Statement  within  the 
overall  DOD-policy  framework. 

The  idea  of  applying  systemic  methods  to  management 
and  resource-allocation  problems  related  to  Defense  is  not 
new.  Since  World  War  II,  quantitative  techniques  have  been 
applied  as  a  viable  means  of  addressing  the  problems 
inherent  to  military  operations  (Morse  and  Kimball,  1951) . 
Quantitative  methods  have  been  applied  to  research  and  develop¬ 
ment  programs  for  fifteen  years  or  more.  Dean  (1963),  Roberts 
(1964),  and  Villers  (1964)  wrote  about  applying  operations 
research  or  scientific  means  to  the  planning,  management, 
and  control  of  research  and  development  programs. 

Models  from  the  current  management-science/operations- 
research  (MS/OR)  could  be  used  to  construct  a  potentially 
useful,  generalized  description  of  the  overall  (macrolevel) 
systems-acquisition  process.  Clearly,  significant  insight 
could  be  gained  into  the  nature  and  structure  of  the  overall 
problem  by  using  such  models.  For  example,  deterministic 
and  stochastic  dynamic  processes  and  related  optimization 


models  could  be  used  to  describe  the  sequential  aspect  of 
the  acquisition  problem  associated  with  the  four  major-decision 
milestones,  decision  analysis  could  incorporate  the  notion 
of  risk  and  uncertainties  inherent  in  R&D  problems,  and 
game-and-team  theory  could  provide  for  the  trade-off  of 
systems-acquisition  programs  competing  for  scarce  resources. 

The  development  of  such  a  model  for  realistic  pre¬ 
scriptive  purposes  would  certainly  involve  substantial  and 
possibly  unresolvable  difficulties.  The  multi-dimensional 
nature  of  DOD  systems-acquisition  problems  and  the  fact 
that  related  functional  relationships  would  often  be,  at  best, 
difficult  to  establish  suggests  that  precise  formulation  of  the 
objectives  function (s)  and  the  concomitant  constraints  would 
be  extremely  difficult.  Furthermore,  the  objectives  and 
constraint  functions  that  could  be  explicitly  stated  would 
likely  inherit  severe  non-linearities  and  discontinuities. 

These  technical  considerations  and  the  obviously  large  size 
of  the  model  imply  that  the  model  would  be  intractable. 

The  discussion  above  is  not  intended  to  discredit 
MS/OR  optimization  techniques  in  their  general  application, 
but  rather,  to  indicate  that  optimization  techniques  would  be 
inappropriately  applied  at  the  macrolevel  of  the  DOD 
systems-acquisition  process  if  actual  solutions  are  being 
sought.  These  techniques  and  others  from  the  MS/OR  area  do 
have  application  to  the  resolution  of  systems-acquisition 
problems,  but  more  appropriately  at  the  level  of  the 
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activities  (the  microlevel)  making  up  the  several  phases  of 
the  systems-acquisition  process .  At  this  level  the  problems 
could  be  more  concisely  stated,  and  the  objective  and  constraint 
functions  could  be  more  readily  formulated,  thereby  enabling 
credible  solutions  to  be  obtained  from  the  optimization  models. 

At  the  macrolevel  of  the  systems-acquisition  process, 
an  aggregate  modelling  approach  is  needed,  which  would  be: 

1.  Broad  enough  to  account  for  the  multivariable 
nature  of  the  process; 

2.  flexible  enough  to  consider  the  inherent  com¬ 
plexities  of  the  problem; 

3.  time  related  to  incorporate  the  several  phases  of 
the  systems-acquisition  process; 

4.  decision  oriented  to  reflect  the  requirements  of 
the  several  major-decision  milestones; 

5.  adaptable  enough  to  incorporate  MS/OR  techniques 
that  could  be  used  more  appropriately  at  the  microlevel 
(activities  level)  of  the  systems-acuqisition  process. 

A  technical  approach  that  embodies  these  characteris¬ 
tics,  which  are  necessary  for  addressing  DOD  svstems- 
acquisition  problems,  is  systems  engineering.^"  Robert  A. 

Frosch  (1969),  formerly  Assistant  Secretary  of  the  Navy  for 

^"Following  Sage  (1977),  the  word  systems  refers  to  the  application 
of  system  science  and  methodologies  associated  with  this  science;  the  word 
engineering  includes  not  only  the  mastery  and  manipulation  of  physical 
data  but  also  implies  social  and  behavioral  considerations  as  inherent 
parts  of  the  engineering-design  process. 
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Research  and  Development,  discussed  the  application  of  systems 
engineering  to  the  systems-acquisition  process.  Specifically, 
he  addressed  the  then  current  practice  of  systems  engineering 
in  relation  to  military  systems  development  and  pointed  out 
several  problems  with  that  practice,  including  the  entrance- 
ment  with  technique  as  a  substitute  for  good  judgment; 
employment  of  serial  versus  iterative  development  models;  the 
fulfillment  of  schedule  predictions  as  a  criterion  to  evaluate 
development  performance  versus  a  set  of  evaluation  factors 
associated  with  the  fulfillment  of  a  need;  the  failure  to 
appropriately  consider  risk;  and  the  optimization  of  the  system 
to  meet  a  very  specifically  defined  operational  requirement. 
Frosch  also  indicated  the  need  to  provide  managers  of  Defense 
systems-acquisition  programs  "a  good  choice  of  managerial  and 
systems-engineering  tools"  to  carry  out  their  responsibilities. 

The  work  of  David  Packard  and  others,  as  related  in 
the  section  Laird/Packard  and  New  Approaches  to  Systems 
Acquisitions  (p.  10) ,  did  much  to  improve  the  managerial 
tools  for  the  Defense  systems-acquisition  process;  particularly 
in  his  efforts  to  strengthen  the  program-management  function 
and  the  establishment  of  the  DSARC.  However,  it  was  the 
author's  perception  that  Frosch 's  proposed  set  of  "good 
systems-engineering  tools"  had  not  yet  evolved  to  assist 
top-level  DOD  managers  in  their  systems-acquisition  decision 
process.  Therefore,  this  research  was  directed  toward  the 


development  of  a  systems-engineering  methodology  to  support 


the  systems-acquisition  decision  process.  In  response  to 
the  problems  noted  by  Frosch,  several  items  were  considered 
in  developing  the  methodology:  the  use  of  techniques  that 
complimented  the  judgment  and  experience  of  top-level  decision 
makers,  the  employment  of  iterative  models  of  the  development 
cycle,  the  establishment  of  a  broad  set  of  criteria  for 
program  evaluation,  the  overt  consideration  of  risk  and 
uncertainty  and  its  impact  on  the  various  phases  of  the 
development  effort,  and  the  pursuit  of  systems-acquisition 
improvement  and  net  necessarily  optimization. 

The  systems-engineering  approach  is  used  in  this 
dissertation  as  the  means  for  improving  the  Defense  systems- 
acquisition  process. 

Purpose  and  Objectives  of  the  Research 

The  application  of  systems  engineering  to  the  Defense 
systems-acquisition  process  is  specifically  addressed  in 
this  dissertation.  This  was  done  with  the  DOD-policy  frame¬ 
work  outlined  in  Chapter  1  and  deals  with  large-scale 
systems-acquisition  problems,  including  application. 

The  following  research  objectives  were  developed: 

1.  To  suggest  conceptual  improvements  in  the  Defense 
systems-acquisition  process; 

2.  To  present  a  validated  systems-engineering 
methodology  to  improve  the  systems-acquisition 


process; 
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3.1  To  develop  a  systems-engineerir.g  metho¬ 
dology  for  the  Defense  systems-acquisition 
process : 

4.1.1  To  examine  the  nature  and  structure 
of  the  systems-acquisition  process; 

4.1.2  To  establish  criteria  for  the  appli¬ 
cation  of  systems-engineering  proce¬ 
dures  to  Defense  systems  acquisition; 

4.1.3  To  gain  insight  into  the  structure  of 
the  value  system  and  procedures  asso¬ 
ciated  with  the  systems-acquisition 
process ; 

4.1.4  To  identify  appropriate  implementa¬ 
tion  procedures  for  the  systems- 
engineering  method  within  the  DOD 
policy  framework. 

3.2  To  demonstrate  the  applicability  of  the 
systems-engineering  framework  to  an  applied 
Defense  systems-acquisition  problem: 

4.2.1  To  analyze  a  development  program  that 
has  benefited  by  the  application  of 
systems-engineering  techniques; 

4.2.2  To  analyze  a  program  that  has  suffered 
from  the  lack  of  a  systems-engineering 
approach . 

w*»  objectives  tree  in  Figure  4  shows  how  the  objectives  at 
«  •  '--‘vel  'Jill  contribute  to  the  accomplishment  of  the  next 
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higher  objective.  The  objectives  become  more  specific  and 
concrete  as  one  moves  down  the  tree. 

The  level-1  objective  is  a  value-laden  objective, 

suggesting  the  possibility  of  improving  the  Defense  svstems- 

acquisition  process.  The  level-2  objective  relates  to  the 

employment  of  a  systems-engineering  methodology  to  improve 

>■  s 

the  process.  Accomplishment  of  this  objective  is  dependent 
on  the  level-3  objectives,  dealing  with  the  development  of  a 
systems-engineering  methodology  and  the  application  of  a 
systems-engineering  methodology  to  an  applied  problem.  The 
several  tasks  of  this  dissertation  were  directly  related  to 
the  level-2  and  -3  objectives. 

Scope  and  Focus  of  the  Study 

This  dissertation  concentrates  on  the  application  of 
systems  engineering  to  the  DOD  systems-acquisition  process. 
However,  it  is  important  to  note  the  following  limitations: 

1.  Policy  C onsiderati ons .  The  investigation  was 
generally  conducted  within  the  policy  framework  established 
by  OMB  Circular  A-109,  DOD  Directives  5000.1  and  5000.2,  and 
the  other  applicable  directives.  The  various  decision  points, 
the  membership  of  decision  groups,  the  specifically  defined 
organizational  relationships  and  procedures  were  usually 
accepted  as  established  by  these  directives.  However,  it  may 
be  appropriate,  based  on  the  research,  to  identify  systems- 
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acquisition-policy  initiatives  that  could  improve  the  overall 
process. 

2.  Organizational  C onsiderations .  The  Department 
of  Defense  is  the  focal  point  of  this  study;  the  relation¬ 
ship  of  DOD  to  Congress  and  agencies  of  the  executive  branch, 
such  as,  the  Office  of  Management  and  Budget  (OMB),  was 
considered  only  as  they  interfaced  with  the  systems-acquisition 
process.  The  detailed  inner  workings  of  these  organizations 
are  not  addressed.  Also,  organizations  within  the  Department 
of  Defense  whose  principal  responsibilities  were  peripheral 

to  the  systems-acquisition  process  were  not  studied  in  depth. 
For  example,  the  intelligence-related  organizations  within 
DOD  were  considered  since  they  provided  enemy-threat 
assessments  for  the  requirements-def inition  problem.  However, 
threat  assessments  were  considered  as  given  inputs  to  the 
systems-engineering  effort,  without  concern  for  their 
specific  development. 

3.  Life  Cycle  of  the  Overall  System.  This  investiga¬ 
tion  covers  the  time  from  when  the  requirement  for  a  new 
mission-element  need  is  first  identified  until  a  final 
decision  is  made  for  the  production  and  deployment  of  a  system. 
The  life  cycle  of  those  phases  of  the  overall  system  dealing 
with  production,  distribution,  operation,  and  retirement  are 
not  addressed. 

4.  Depth  of  Analysis .  This  study  is  primarily 
concerned  with  the  nature  and  structure  of  the  systems- 
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acquisition  problem.  Each  phase  of  the  development  effort 
was  evaluated  to  see  what  activities  needed  to  be  performed 
and  how  the  activities  could  best  be  represented  or  analyzed. 
Considerations  were  developed  for  applying  systems-engineering 
tools  and  techniques  to  the  various  steps  of  each  phase. 

These  considerations  were  then  used  to  identify  where 
in  the  systems-acquisition  process  the  various  tools  and 
techniques  of  systems  engineering  could  best  be  applied. 
However,  detailed  mathematical  models  were  not  formulated  in 
response  to  specific  systems-acquisition-analysis  requirements. 

5.  Program-Manag  ement  Considerations.  The  overall 
management  of  systems  acquisition  is  considered  on  the  DOD 
and  the  service-headquarters  levels.  Program  management  is 
addressed  from  the  program  manager's  viewpoint,  concentrating 
primarily  on  his  relationships  and  responsibilities  to  the 
DSARC/ (S) SARC. 

Emphasis  is  placed  on  the  major-decision  milestones 
(discussed  in  Chapter  1)  as  beginning  and  end  points  of 
each  phase  of  the  systems-acquisition  process — i.e.,  each 
milestone  represents  the  end  of  the  preparations  and  analysis 
steps  leading  to  the  decision  and  the  beginning  of  the 
next  phase  of  the  systems-acquisition  process.  By  making 
the  decision  milestones  the  focal  point  of  the  study  effort, 
operations  and  responsibilities  of  the  DSARC/ (S) SARC  are 
highlighted.  These  councils  are  one  of  the  principal  means 
by  which  DOD  has  implemented  the  systems-acquisition  policy. 
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Therefore,  this  dissertation  emphasizes  the  following  aspects 
of  the  DSARC/  (S)  SARC  :  the  personnel  makeup  of  the  decision 
groups,  the  value  systems  operating  within  the  council,  the 
council  proceedings,  the  decision  process  of  the  members,  and 
the  impact  of  the  decisions  on  the  overall  systems-acquisition 
process. 


Comments  on  Related  Literature 

A  great  deal  of  material  in  the  literature  was 
related  t p  the  general  area  of  research  and  development.  This 
material  was  principally  centered  on  three  areas:  (1)  The 
selection  and  evaluation  of  research  or  development  projects; 
(2)  the  use  of  management-engineering  techniques  in  research 
and  development;  and  (3)  the  management  of  development 
projects. 

In  the  first  area,  wide-ranging  methods  for  the 
evaluation  and  selection  of  projects  were  discussed  in  survey 
articles  by  Cetron,  Martino,  and  Roepche  (1967);  Augood 
(1973) ;  Baker  (1974) ;  Clarke  (1974) ;  and  Baker  and  Freeland 
(1975) .  The  Cetron,  Martino,  and  Roepche  article  addressed 
the  application  of  quantitative  methods  to  the  selection 
of  R&D  programs.  It  presented  the  characteristics  of  the 
methods,  commented  on  their  ease  of  use,  and  classified  the 
methods  by  type;  i.e.,  operations  research,  economic  analysis, 
or  decision  theory.  The  Augood  article  documented  a  variety 
of  R&D-evaluation  methods  and  associated  techniques. 
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Checklists,  various  R&D  indices  (i.e.,  risk  analysis, 
decision  analysis,  and  impact  analysis),  and  combined  methods 
were  discussed. 

In  his  article.  Baker  described  the  R&D  project 
selection  problem  as  consisting  of  the  requirement  to  generate 
alternatives,  to  determine  the  need  for  a  decision,  to  collect 
data,  to  specify  constraints  and  criteria,  and  to  recycle. 

He  assessed  the  existing  R&D  benefit-measurement  and  project- 
selection  literature  and  discussed  the  limitations  of  the 
various  models.  Clarke's  (1974)  paper  discussed  the  factors 
to  be  considered  when  a  manager  has  to  decide  whether  a  concept, 
product,  or  system  is  to  go  to  the  next  stage  of  the  develop¬ 
ment  process.  The  literature  reviewed  covered  the  following 
topics:  idea  generation,  transmission  and  evaluation,  project 
selection  and  evaluation,  and  the  impact  of  project-selection 
models  on  scientists  and  engineers.  Several  classes  of 
mathematical  models  to  be  used  for  project  selection  and 
evaluation  were  also  presented. 

The  Baker  and  Freeland  (1975)  paper  assessed  the 
literature  addressing  quantitative  models  for  R&D  project 
selection  and  resource-allocation  decisions.  The  literature 
surveyed  was  dichotomized  into  benefit-measurement  methods  and 
resource-allocation  methods.  The  paper  discussed  the  strengths 
and  limitations  of  existing  methods  and  emphasized  empirical 
investigations.  A  variety  of  methods  v/as  discussed,  including 
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decision  analysis,  scoring  models,  zero-one  approaches, 
and  mathematical  programming  models. 

Management  engineering  and  its  application  to  research 
and  development  has  also  received  a  good  deal  of  attention 
in  the  literature.  An  excellent  example  is  Warfield  and 
Hill's  (1971)  paper  dealing  with  the  portrayal  of  R&D  projects 
through  DELTA  charts.  DELTA  charts  may  be  used  to  describe 
not  only  events  and  activities  but  also  decision  and  logic 
function.  The  paper  pointed  out  the  critical  need  to  portray 
these  functions,  which  are  so  important  in  representing 
the  alternate  approaches  and  feedback  loops  that  are  integral 
to  R&D  project  planning. 

The  Work  Plan  Analysis  and  Scheduling  Technique 
(WoPAST)  was  developed  by  Delgrosso  and  Rosenbluth  (1976) 
to  plan  and  schedule  complex  engineering  programs-  when 
early  ideas  and  goals  were  still  in  a  state  of  flux  and 
responsibilities  were  general  and  not  clearly  defined.  Their 
paper  pointed  out  that  the  object  of  WoPAST  as  a  tracking 
and  scheduling  device  is  to  define  activities,  measure 
progress,  and  react  to  problems.  WoPAST  merged  the  procedures 
of  the  program  evaluation  and  review  technique/critical  path 
method  (PERT/CPM)  with  communications  and  feedback  techniques 
appropriate  to  the  organization  of  the  user. 

A  management-engineering  technique  that  has  been 
applied  to  R&D  was  originally  developed  by  Pritsker  and  Happ 
(1966) ,  and  Pritsker  and  Whitehouse  (1966)  .  Moore  and  Taylor 
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(1977)  used  this  technique  to  address  multiteam,  multiproject 
research  and  development  planning.  Specifically,  this  paper 
reported  on  a  simulation  study  of  multiple  research  and  develop¬ 
ment  projects  that  were  investigated  concurrently  and  sequen¬ 
tially  by  more  than  one  research  team.  GERT  was  suggested 
for  use  because  of  its  capability  to  incorporate  the  probabi¬ 
listic  outcomes  and  feedback  loops  that  are  common  to  R&D 
projects. 

The  papers  discussed  above  are  not  intended  to  be 
a  comprehensive  review  of  the  management-engineering  tech¬ 
niques  that  can  be  applied  to  research  and  development. 

Rather,  these  papers  should  be  viewed  as  representative  of 
the  type  of  research  taking  place  in  this  area. 

The  third  major  area  of  extensive  work  and  publi¬ 
cation  related  to  R&D  was  project  management.  A  variety 
of  texts  and  manuals  have  discussed  the  area  of  project 
management  in  great  depth. 

Blanchard  (1974,  1976,  1978)  did  an  excellent  job 
of  setting  forth  the  systems  life-cycle  problems.  Among 
other  topics,  he  covered  planning,  development,  product 
research,  design,  test  and  evaluation,  production,  and  life- 
cycle  costs.  His  work  was  particularly  significant  in  the  con¬ 
text  of  this  dissertation  since  much  of  it  was  related  to 
Defense  R&D. 

In  his  text  on  the  management  of  systems  engineering, 
Chase  (1974)  discussed  a  wide-ranging  set  of  topics,  including 


the  system  design,  management  organizations,  operational 
considerations,  testing  and  evaluation,  software  engineering, 
dealing  with  intangibles,  and  risk  assessment.  The  Depart¬ 
ment  of  the  Navy's  Naval  Ocean  Systems  Center,  San  Diego 
(1977),  recently  published  a  Project  Manager's  Guide  for 
use  by  developers  of  naval  systems.  It  addressed  the 
chronology  of  systems  development  and  the  key  disciplines 
involved.  Since  this  document  dealt  almost  exclusively  with 
Defense  development  programs,  it  was  highly  relevant  to  the 
programs  considered  in  this  dissertation. 

Shinners  (1976)  directed  his  text  toward  the  systems 
engineers  and  program  managers  working  on  large,  technologi¬ 
cally  complex  programs.  He  noted  the  need  for  interdiscipli¬ 
nary  teams  and  effective  communications  techniques.  Shinners 
also  discussed  systems  engineering  and  systems  management  from 
both  the  practical  and  analytical  viewpoints.  In  his  text, 
Archibald  (1976)  developed  a  generic  perspective  of  project 
management  generally  applicable  to  many  tasks.  Although 
written  primarily  for  project  managers,  this  text  also 
contained  many  useful  insights  for  those  who  manage  project 
managers  and  those  who  support  project  managers  in  functional 
and  staff  departments.  The  book  is  divided  into  two  major 
parts:  an  executive  guide  to  program  management  and  the 
management  of  specific  projects.  Coutinho  (1977)  addressed 
the  development  management  of  advanced  systems.  He  covered 
many  topics;  including  procurement,  contracting,  design 
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assurance,  organization  for  management,  quality  assurance, 
operational  testing,  and  professional  responsibilities.  This 
text  was  also  oriented  towards  Defense  systems  development. 
Hajek's  (1977)  book  presented  the  various  facets  of  project 
management,  chronologically  covering  the  life  cycle  of  a 
system  from  the  time  a  company  took  an  interest  in  the 
project  until  the  system  was  accepted  and  support  was  estab¬ 
lished.  Throughout  the  book,  the  procurement  of  a  land-mass 
simulator  was  used  as  the  vehicle  for  illustrating  the 
various  principles  and  disciplines  involving  a  project 
engineer . 

Collectively,  these  several  references  dealt  in 
detail  with  various  aspects  and  functions  of  project  management. 
However,  the  dissertation  deals  with  this  subject  only  in  a 
general  way  and  emphasizes  the  upper  levels  of  the  hierarchy 
of  systems-acquisition  management. 

The  three  general  areas  of  R&D  technical  literature 
discussed  above  were  the  selection  and  evaluation  of  research 
or  development  projects,  management-engineering  techniques, 
and  development-project  management.  The  work  in  the  first 
area  focused  on  the  relatively  narrow  aspects  of  specific 
decision  points  associated  with  R&D  projects.  The  second 
area  of  work  addressed  only  certain  project  facets,  which  were 
a  subset  of  the  overall  systems-acquisition  process.  Finally, 
the  project-management  texts  tended  to  accept  the  problem 
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as  reasonably  well  defined  and  generally  did  not  touch  upon 
policy  considerations. 

The  need  for  an  enlightened,  comprehensive  approach 
to  the  management  problem  of  Defense  systems  acquisition 
must  be  considered.  Two  recent  articles  suggested  the  need 
for  a  broad  systems  approach  to  research  and  development. 

Paul  Polishuk  (1976)  identified  the  need  for  an  interdiscipli¬ 
nary  approach  to  policy  analysis  and  problem  solving  due  to 
the  increasing  complexities  of  government  policy  and  decision 
making.  John  W.  Lathrop  and  Kan  Chen  (1976)  wrote  of  the 
relevance  of  systems  theory  to  the  evaluation,  comparison, 
and  planning  of  R&D  programs  with  diverse  objectives, 
uncertain  outcomes,  and  multidimensional  benefits  and  costs. 
Both  articles  noted  the  need  for  a  systems-engineering 
framework  within  which  government  research  and  development 
programs  could  be  carried  out.  In  like  manner,  Lawrence  S. 
Hill  (1970)  presented  a  historical  analysis  of  the  early 
use  of  certain  aspects  of  a  systems-engineering  approach 
to  government  R&D  programs  and  commented  on  the  benefits 
of  such  an  approach. 

The  application  of  systems  engineering  to  federal 
R&D  and,  specifically,  to  the  Defense  systems-acquisition 
process  is  the  subject  of  the  research  associated  with  this 


dissertation . 


Organization  of  the  Report 


The  purpose  of  the  research  presented  in  this  disser 
tation  is  to  address  the  question  of  the  application  of 
systems  engineering  to  military  systems  acquisition  within 
the  DOD-policy  framework  previously  discussed.  The  actual 
research  was  divided  into  three  major  tasks.  Each  task 
is  covered  in  a  self-contained  chapter,  which  includes  the 
references  appropriate  to  that  chapter. 

The  DOD  systems-acquisition  material  for  the  report 
v/as  developed  from  three  sources;  DOD  directives  and  other 
documentation,  interviews  with  DOD  personnel  involved  with 
systems  acquisition  (Appendix  B) ,  and  the  author's  personal 
experiences  in  four  years  of  Defense  R&D .  A  questionnaire 
for  those  personnel  interviewed  is  presented  as  Appendix  C. 

The  first  major  task  was  to  analyze  the  structure 
of  the  problems  of  Defense  systems  acquisition.  A  general 
systems-engineering  framework  was  adopted  to  analyze  the 
problems  and  the  underlying  structure  of  the  Defense  systems 
acquisition  process  was  examined.  The  systems-engineering 
framework  and  the  systems-acquisition  process  were  compared 
on  a  step-by-step  basis  to  determine  the  correlation  between 
the  two  procedures. 

The  second  major  task  concerned  the  development  of 
a  systems-engineering  methodology  for  application  to  the 
Defense  systems-acquisition  process.  The  nature  and 


57 


structure  of  the  systems-acquisition  process  were  examined, 
considerations  for  the  application  of  the  systems-engineering 
tools  and  techniques  to  the  systems-acquisition  process 
were  identified,  and  a  systems-engineering  methodology  was 
developed  to  support  the  decision  process.  The  implementa¬ 
tion  of  this  methodology  by  the  Department  of  Defense  is 
discussed . 

The  third  major  task  was  to  analyze  an  on-going, 
real-world,  military  systems-acquisition  problem  using  a 
systems-engineering  approach.  Initially,  this  program  was 
examined  when  it  suffered  from  the  lack  of  a  systems- 
engineering  approach.  The  same  program  was  then  examined 
to  see  if  and  how  it  benefited  by  applying  a  systems- 
engineering  method  similar  to  that  developed  in  the  previous 
task. 

The  final  chapter  is  devoted  to  the  summary, 
conclusions,  and  recommendations  for  follow-on  research. 
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Chapter  3 


THE  STRUCTURE  OF  SYSTEMS-ACQUISITION  PROBLEMS 
OF  THE  DEPARTMENT  OF  DEFENSE 

Introduction 

The  multidiscipline  approach  of  systems  engineering 
has  been  a  reality  for  almost  forty  years.  '  A.  D.  Hall  (1962) 
indicated  that  the  initial  use  of  the  term  systems  engineering 
might  be  attributed  to  the  Bell  Telephone  Laboratories  during 
the  early  1940 's  when  they  used  a  systems  approach  to  develop 
the  Bell  network.  Lawrence  S.  Hill  (1970)  identified  the 
Glenn  L.  Martin  Company  as  an  early  practitioner  of  systems 
engineering,  for  the  formation  of  their  Special  Weapons  Group 
in  1945.  Hill  cited  Martin  Company  literature  on  the  Special 
Weapons  Group  that  stated  "the  problems  we  face  in  progress¬ 
ing  into  the  future  consist,  first  of  all,  in  devising  ways 
and  means  for  expanding  the  application  of  the  systems- 
engineering  concept." 

The  military-oriented  work  of  Martin's  Special  Weapons 
Group  and  the  work  of  the  California  Institute  of  Technology's 
Jet  Propulsion  Laboratory  (JPL)  for  the  U.  S.  Army  Air  Corps 
are  early  examples  of  systems-engineering  work  supporting 
Defense-related  projects.  During  World  War  II  the  JPL  supported 
efforts  to  apply  the  principles  of  rocket-motor  design  to 
aircraft  (Pickering,  1973)  .  Towards  the  end  of  the  war, 
when  Army  Ordinance  asked  the  JPL  to  explore  the  possible 
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use  of  rockets  in  long-range  artillery,  rocket  motors,  fuel 
tanks,  guidance  systems,  and  payload  all  had  to  be  considered. 
Therefore,  engineers  from  a  variety  of  disciplines  were  forced 
to  work  together  and  a  very  crude  systems-engineering  approach 
slowly  emerged. 

Next,  the  JPL  developed  an  operational  missile 
system  (Corporal)  for  Army  Ordinance.  Many  different  problems 
arose  in  developing  Corporal  that  were  solved  by  a  team  of 
engineers  from  several  disciplines.  These  problems  included 
defining  operational  requirements,  identifying  constraints, 
establishing  goals,  integrating  systems,  communicating  with 
users  and  sponsors,  and  transferring  the  necessary  technology 
to  the  industrial  company  producing  Corporal.  The  solution 
of  these  problems  led  to  the  evolution  of  a  meaningful  systems- 
engineering  organization  at  JPL,  which  further  matured  through 
the  total  systems  responsibility  JPL  was  given  to  develop  the 
second-generation  Sergeant  missile  during  the  1950's.  Develop¬ 
ment  of  this  missile  was  a  classical  systems-engineering  task 
for  a  large  and  complex  system  and  was  successfully  completed 
just  as  the  JPL  was  transferred  to  NASA  in  1953  (Pickering, 
1973)  . 

The  Apollo  project  carried  out  by  NASA  during  the  1960's  was 
a  natural  extension  of  the  pioneering  systems-engineering 
efforts  carried  out  by  JPL  and  others  in  the  previous  decade. 
This  was  the  largest  engineering  enterprise  ever  undertaken, 
involving  an  expenditure  of  $20  billion  and  the  efforts  of 
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more  than  200,000  people.  George  E.  Mueller  (1973),  Associate 
Administrator  for  NASA's  Manned  Space  Flight  from  1963  to 
1969  stated  that  "the  [Apollo]  program. .. suggests  the  necessity 
for  creativity  and  innovation  in  modern  systems  engineering." 
This  statement  and  Mueller's  discussion  of  goal-setting  proce¬ 
dures,  the  development  of  operational  requirements,  and  the 
requirement  for  systems  tradeoffs  are  strong  recommendations 
for  using  viable  systems-engineering  techniques. 

One  of  the  pioneers  of  systems  engineering  during 
the  1950 's  and  1960's  was  Simon  Ramo.  He  was  the  chief 
scientist  for  the  Air  Force  on  the  intercontinental  ballastic- 
missile  program,  coordinating  the  scientific  efforts  of 
several  firms  in  this  large  systems-acquisition  program. 

Ramo,  the  organizer  of  the  electronic  and  missile  operations 
for  the  Hughes  Aircraft  Company,  was  also  a  cofounder  of 
Ramo  and  Wooldridge,  which  later  merged  into  TRW,  Inc.  His 
association  with  these  companies  and  his  two  books  applying 
science  to  social  problems  (Ramo,  1969,  1970)  placed  him  in 
the  forefront  of  systems-engineering  associated  with  compu¬ 
ters,  missile  systems,  electronics,  transportation,  and  city 
planning  (Miles,  1973)  . 

While  NASA  was  busy  with  the  Apollo  program  during 
the  1960 's,  the  Department  of  Defense  (DOD)  was  developing 
a  renewed  interest  in  systems  acquisition.  Part  of  this 
interest  was  generated  by  the  planning,  programming,  and 
budgeting  system  (PPBS)  that  Secretary  of  Defense  Robert  S. 
McNamara  and  Dr.  Charles  J.  Hitch  had  issued  (Fabian,  1977). 


The  PPBS  required  that  military-force  components  be  linked 
directly  to  the  resources  required  to  support  them.  This 
linkage  caused  an  increased  emphasis  on  mission-oriented 
operational  requirements  and  thereby  effected  the  systems- 
acquisition  process.  At  the  same  time,  a  variety  of  problems 
and  deficiencies  were  identified  in  the  DOD  systems-acquisition 
process  (Peck  and  Scherer,  1962) . 

In  the  mid-1960's,  in  response  to  the  PPBS  requirements 
and  to  the  problems  identified  with  systems  acquisition,  the 
Air  Force's  principal  development  command,  the  Air  Force 
Systems  Command  (AFSC) ,  published  the  375  series  manuals, 
which  set  forth  the  management  practices  to  be  used  in 
systems  acquisition.  Systems  Program  Office  Manual  (USAF, 

1966c)  provided  an  overview  and  an  introduction  to  the  manage¬ 
ment  of  the  systems-acquisition  process.  Systems  Program 
Management  Procedures  (USAF,  1966b)  established  the  overall 
framework,  procedures,  and  objectives  for  the  systems- 
acquisition  life  cycle.  But  it  was  the  third  manual.  Systems 
Engineering  Management  Procedures  (USAF,  1966a),  that  set  forth 
an  overall  systems-engineering  approach  for  systems  acquisition, 
which  included  defining  the  problem,  establishing  the  opera¬ 
tional  requirements,  setting  the  objectives,  managing  the 
overall  process,  periodically  reviewing  the  designs,  communi¬ 
cating,  and  testing  and  evaluating.  The  manual  stated  that 
developing  a  system  was  a  multidiscipline  task  and  that  these 
various  disciplines  needed  to  be  integrated  into  v/hat  is  now 
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called  systems  teams.  The  following  phases  of  the  systems  life 
cycle  were  identified  in  this  third  manual:  conceptual,  defini¬ 
tion,  acquisition,  and  operational  phases.  Although  this  manual 
did  not  formulate  a  systems-engineering  methodology,  the  concepts 
of  a  systems-engineering  approach  were  clearly  outlined. 

All  the  services  have  continued  to  pursue  their 
systems-engineering  interests  as  they  relate  to  DOD  systems 
acquisition.  The  Department  of  the  Navy ' s  -Weapon  Systems 
Selection  and  Planning  (Navy.,  1974)  and  the  Air  Force's 
Engineering  for  Defense  Systems  (USAF,  1976)  are  examples 
of  the  services'  continued  interest  in  an  overall  systems- 
engineering  approach  to  systems  acquisitions. 

The  above  discussion  points  out  the  historical 
precedent  for  a  relationship  between  the  Defense  systems- 
acquisition  process  and  systems  engineering.  Such  procedures 
as  those  set  forth  in  Systems  Engineering  Management  Procedures 
(USAF,  1966a)  evolved  from  the  military  services,  extensive 
experience  in  developing  and  managing  highly  complex, 
large-scale  systems.  The  military  services  have  been  in  the 
forefront  of  those  using  these  types  of  large-scale,  complex 
systems  and  have  dealt  with  them  in  a  holistic  fashion  (Hill,  1970). 

In  this  chapter,  the  Defense  systems-acquisition 
process  is  discussed  in  the  context  of  a  systems-engineering 
framework  and  from  the  standpoint  of  the  DOD-policy  directives 

^Hill  defined  complex  as,  ”[A  system  in  which]  a  change  in  one 
variable  will  affect  many  other  variables  in  the  system,  rarely  in  linear 
fashion.  An  equally  important  characteristic  is  the  existence  of  multiple 
feedback  loops  and  feedback  loops  within  feedback  loops." 
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previously  mentioned.  Those  areas  wherein  the  actual  pro¬ 
cedures  used  in  the  systems-acquisition  process  vary  within 
the  DOD-policy  framework  established  by  the  directives  are 
taken  up  in  the  next  chapter. 

After  reviewing  the  existing  systems-engineering 
approaches,  a  specific  systems-engineering  framework  is  used 
throughout  the  remainder  of  this  dissertation.  The  DOD-policy- 
directed  approach  to  the  systems-acquisition  process  is  then 
examined  to  clearly  identify  its  functions,  activities,  and 
decision  points.  Finally,  a  comparative  analysis  of  the 
selected  steps  and  phases  of  a  systems-engineering  framework 
and  the  activities  and  phases  of  the  Defense  systems- 
acquisition  process  demonstrate  the  relationship  between 
the  two  procedures. 

The  Systems -Engineering  Framework 

The  following  definitions  of  systems  engineering  and 
its  purpose  are  suitable  for  use  with  problems  of  this  scope 
and  complexity: 

Systems  engineering  is  an  appropriate  combination  of  the 
mathematical  theory  of  systems  and  behavioral  theory  in  a  use¬ 
ful  setting  appropriate  for  the  resolution  of  real  world  problems. 

The  purpose  of  systems  engineering  is  to  develop  policies  for  the 
management,  direction,  and  regulation  activities  relative  to  the 
planning,  development,  production,  and  operation  of  total 
systems  to  maintain  overall  integrity.  (Sage,  1977c) 

Although  these  definitions  can  be  broadly  applied  to  many 
systems  and  policy  approaches,  the  Defense  systems  and  the 
DOD  policy  framework  governing  their  acquisition  are  speci¬ 
fically  addressed  here. 


Jfl 


68 


A  methodology  is  "an  open  set  of  procedures  which  pro¬ 
vides  the  means  for  solving  problems"  (Sage,  1977a);  a  set  of 
tools,  a  set  of  proposed  activities  for  problem  solution,  and  a 
set  of  relations  among  the  tools  and  activities  constitute  a 
methodology.  The  tools  of  systems  engineering  are  words, 
mathematics,  and  graphics.  These  tools,  including  algorithms 
and  concepts,  enable  various  activities  within  systems 
engineering  to  be  carried  out.  The  particular  sets  of 
relations  among  tools  and  activities  that  constitute  the 
framework  (procedural  structure)  for  systems  engineering  are 
of  particular  importance  because  such  a  framework: 

1.  Does  a  great  deal  to  enable  the  handling  of 
the  considerations,  interrelations,  and  controversial  value 
judgments  of  large-scale,  complex  problems; 

2.  is  of  significant  value  in  teaching  the  policy 
makers  and  decision  makers  the  value  of  the  systems  approach; 

3.  allows  problems  to  be  resolved  at  the  institutional 
and  value  levels  as  well  as  at  the  symptomatic  level  where  so 
much  effort  is  concentrated. 

For  a  systems-engineering  methodology  to  be  developed 
within  a  systems-engineering  framework,  the  framework  should 
have  the  following  characteristics: 

1.  It  must  be  decision  oriented  to  meet  the  demands 
of  decision  makers. 

2.  It  must  be  broad  enough  to  cover  the  life-cycle 
phases  of  various  systems  from  conception  of  a  need  to 
retirement  of  the  system. 
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3.  It  must  consider  the  multidiscipline  nature  of 
systems  problems. 

4.  It  must  incorporate  the  value  system  operative 
in  each  problem. 

5.  It  must  be  adaptable  and  flexible  enough  to 
have  general  application  to  systems-engineering  problems. 

The  last  characteristic  is  important  as  it  infers  the 
need  for  a  standard  systems-engineering  framework  to  be 
applied  to  large-scale,  complex  problems.  Some  systems- 
engineering  methodologies  were  discussed  in  the  following 
sources:  Arthur  D.  Hall,  III  (1962,  1969),  Harold  Chestnut 
(1967),  Lawrence  S.  Hill  (1970),  John  E.  Gibson  (1972,  1973, 
1977a,  1977b),  Ralph  F.  Miles,  Jr.  (1973),  Wilton  P.  Chase 
(1974) ,  and  Vincent  P.  Luchsinger  and  V.  Thomas  Dock  (1976) . 

Other  methodologies  were  referenced  by  Sage  (1977a)  in  an 
editorial  in  the  IEEE  Transactions  on  Sy  stems,  Man ,  and  Cybernetics . 

In  his  editorial,  Sage  discussed  the  need  for  a 
standard  framework  for  systems  engineering.  He  persuasively 
argued  the  merits  of  this  approach  and  effectively  pointed 
out  the  problems  that  would  accrue  to  systems  engineering 
if  a  single  methodology  were  not  adopted.  Sage  further 
proposed  the  adoption  of  the  comprehensive  three-dimensional 
framework,  or  morphology,  set  forth  by  Arthur  D.  Hall,  III  (1969)  , 
as  the  framework  within  which  systems  engineering  should  be 
practiced.  Three  dimensions  of  this  morphology  are  time, 
logic,  and  knowledge. 
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The  time  dimension,  often  referred  to  as  the  course 
structure  of  the  morphology,  depicted  a  sequence  of  activities 
in  the  life  of  the  system  from  its  inception  to  retirement 
and  was  segmented  into  intervals,  or  phases,  by  a  set  of 
major-decision  milestones.  The  various  phases  of  a  system 
life  cycle  were  (Hall,  1962,  1969): 

1.  Program  Planning :  the  organization  identified  the 
activities  and  projects  it  would  study  on  iriore  detailed  levels 
of  planning.  The  program  was  normally  large  scale,  complex, 
and  long  term. 

2.  Project  Planning:  the  organization  "concentrated 
on  one  of  the  several  projects  of  the  overall  program. 

3.  System  Development:  planning  was  implemented; 
this  phase  dealt  primarily  with  components  rather  than  with 
overall  alternatives  and  ended  by  preparing  detailed 
specifications,  drawings,  and  bills  of  materials  for  the 
manufacturers  or  construction  organization. 

4.  Production :  all  required  activities  gave  physical 
embodiment  to  the  system.  For  a  new  product,  the  manufacturing 
engineers  determined  the  sequence,  material  flow,  and  floor 
layout  required,  designed  the  tooling  and  test  jigs,  and 
established  quality  control. 

5.  Distribution :  the  system  was  phased-in 

and  conveyed  to  the  ultimat  users.  This  could  involve  all 
kinds  of  distribution  facilities,  preparation  for  support 
functions,  and  training  for  systems  maintenance  and  opera¬ 
tion  personnel. 
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6.  Operations :  the  system  was  used  or  the  product 
consumed.  The  operational  period  was  the  reason  for  all 
forms  of  systems  engineering. 

7.  Retirement :  the  system  was  taken  out  of  service 
or  the  product  phased-out. 

The  second  dimension  of  the  systems-engineering 
morphology  was  a  problem-solving  procedure  that  had  several 
steps  to  be  carried  out  during  each  phase  of  the  system’s 
life  cycle.  These  steps  were  normally  conducted  in  the 
following  sequence  (Sage,  Warfield,  Thissen,  et  al.  ,  1978): 

1.  Problem  Definition :  the  needs,  constraints,  r>.nd 
alterables  associated  with  an  issue  were  described  and 
standardized.  The  purpose  of  problem  definition  was  to 
clarify  the  issues  under  consideration  so  that  other  steps 
of  the  systems  process  could  be  carried  out. 

2.  Value  Systems  Design:  the  objectives  and  objec¬ 
tives  measures  were  determined,  as  well  as  the  interrelation¬ 
ship  between  objectives  and  between  objectives  and  objectives 
measures  and  the  interaction  between  objectives  and  the 
elements  of  the  problem-definition  step. 

3.  System  Synthesis :  the  policies  or  systems  that 
might  satisfy  the  established  need  or  attain  the  desired 
objectives  were  established.  Normative,  behavioral,  and 
technological  constructs  and  the  transition  scenarios  for 
policy  alternatives  were  delineated. 

4.  Systems  Analysis  and  Modelling :  models  were 
constructed  to  determine  the  consequences  of  pursuing  policies. 


i.e.,  to  determine  the  behavior  or  subsequent  conditions 
resulting  from  alternative  policies  and  systems. 

5.  Optimization  of  Each  Alternative :  the  individual 
policies  or  systems  were  fine  tuned  often  by  adjusting 
parameters  to  refine  each  individual  policy  or  system. 

6.  Decision  Making:  systems  or  policies  were 
selected  (or  not  selected)  for  further  planning  and  resource 
allocation. 

7.  Planning  for  Action:  implementation  efforts, 
resource  and  management  allocations,  or  plans  for  the  next 
phase  of  effort  were  delineated. 

The  third  dimension  referred  to  the  body  of  knowledge 
(facts,  models,  and  procedures)  used  to  define  a  discipline, 
profession,  or  technology.  Possible  disciplines  that  could 
be  employed  within  this  framework  included  lav;,  medicine, 
engineering,  and  the  social  sciences. 

A  single  systems-engineering  framework  is  used  through¬ 
out  this  study.  The  three-dimensional  morphology  box  set 

forth  by  Arthur  D.  Hall,  III  (1969)  is  adopted  as  the  particu- 

2 

lar  approach  to  be  applied  (Figure  5) . 

Sy stems- Acquisition  Policy  and  Milestones 

The  federal  government's  procurement  policies  for  its 
executive-branch  agencies  were  set  forth  by  Office  of  Manage¬ 
ment  and  Budget  (OMB)  in  Major  Systems  Acquisition  (0MB,  1976) 

2 

This  morphology  box  will  be  referred  to  throughout  the 
dissertation  as  the  systems-engineering  framework. 
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wherein  a  major  system  was  defined  as 

a  combination  of  elements  that  will  function  together  to 
produce  the  capabilities  required  to  fulfill  a  mission  need 
(i.e.,  a  required  capability  within  an  agency's  overall  purpose). 

To  be  defined  as  a  major  systems-acquisition  program  a 
program  must  be  directed  at  and  critical  to  fulfilling  an 
agency  mission,  must  entail  the  allocation  of  relatively 
large  resources,  and  must  warrant  special  management  atten¬ 
tion.  The  circular  also  provided  for  the  agency  head  to 
establish  relative  dollar  thresholds  as  one  of  the  criteria 
for  determining  which  programs  should  be  considered  as  a 
major  systems-acquisition  program. 

The  policies  of  the  circular,  which  were  developed  to 
assure  the  effectiveness  and  efficiency  of  the  process  of 
acquiring  major  systems,  are: 

1.  To  express  needs  and  program  objectives  in 
mission  terms  and  not  equipment  terms,  to  encourage  innova¬ 
tion  and  competition  in  creating,  exploring,  and  developing 
alternative  systems-design  concepts. 

2.  To  emphasize  the  initial  activities  of  the  systems- 
acquisition  process  to  stress  the  competitive  exploration  of 
alternative  systems  in  response  to  mission  needs.  Whenever 
economically  beneficial,  the  competition  between  similar 

or  differing  systems  designs  should  continue  throughout 
the  program. 

3.  To  inform  Congress  of  a  major  systems-acquisition 
program  immediately  after  the  agency  head  has  approved  a 


new  mission  need. 
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4.  To  establish  clear  lines  of  authority,  responsi¬ 
bility,  and  accountability  for  the  management  of  a  major 
systems-acquisition  program  by  using  the  appropriate  managerial 
levels  in  decision  making  and  obtaining  agency-head 

approval  at  key  decision  points. 

5.  To  appoint  an  acquisition  executive  to  be  res¬ 
ponsible  for  integrating  and  unifying  the  systems-acquisition 
management  process  and  implementing  a  monitoring  policy.  This 
appointment  is  to  be  made  by  the  agency  head. 

6.  To  establish  a  systems-acquisition  plan  built 
on  an  analysis  of  agency  mission  needs  and  an  appropriate 
allocation  of  resources. 

7.  To  tailor  an  acquisition  strategy  for  any  program, 
as  soon  as  the  agency  decides  to  solicit  alternative  systems- 
design  concepts,  that  could  lead  to  the  acquisition  of  a  new 
major  system  and  to  refine  the  strategy  throughout  the 
acquisition  process. 

8.  To  designate  a  program  manager  for  each  major 
systems-acquisition  program  as  soon  as  a  decision  is  made  to 
fulfill  a  mission  need  by  pursuing  alternative  systems-design 
concepts . 

Technical  and  program  decisions  would  normally  be 
made  at  the  level  of  the  agency  component  or  operating 
activity  (program  office) .  However,  the  policies  of  Major 
Systems  Acquisition  (OMB,  1976) provided  for  the  agency  head 
to  have  control  over  the  agency's  major  acquisitions  by 


reserving  for  him  the  right  to  make  the  key  decisions 
during  the  systems-acquisition  process.  These  four  key 
decisions  were: 

1.  The  approval  of  a  specifically  identified  and 
defined  mission  need ,  of  the  relative  priority  assigned 
within  the  agency,  and  of  the  general  magnitude  of  resources 
that  may  be  invested. 

2.  The  selection  of  competitive  systems-design 
concepts  to  be  advanced  to  a  test  or  demonstration  phase 
or  the  authorization  to  proceed  with  the  development  of  a 
noncompetitive  (single-concept)  system. 

3.  The  commitment  of  a  system  to  full-scale  develop¬ 
ment  and  limited  production, 

4.  The  commitment  of  a  system  to  full  production. 

Each  decision  is  a  go  or  no-go  decision  and  requires 

the  reaffirmation  of  the  prior  decision.  These  decisions 
were  designed  to  establish  concise  and  well-defined  mission 
needs,  to  maximize  the  number  of  viable  system  alternatives 
considered,  and  to  enhance  the  design  and  planning  efforts. 

In  making  these  key  decisions,  the  agency  head  exerts 
meaningful  control  over  the  direction  and  scope  of  the  systems- 
acquisition  programs  within  his  agency.  Using  the  procedures 
of  Warfield  and  Hill  (1971) ,  a  DELTA  Chart  alternating  phases 
of  the  systems-acquisition  process  with  the  key  decisions  by 
the  agency  head  is  presented  in  Figure  6. 


The  policies  of  the  OMB  directive  (1976)  were  effected 
by  two  revised  DOD  directives,  Major  Systems  Acquisition , 

(DOD,  1977a)  and  Major  Systems  Acquisition  Process,  (DOD, 

1977b) .  Both  directives  reiterated  the  policies  of  the  OMB 
paper  and  tailored  those  policies  to  the  DOD.  Many  policies 
introduced  by  the  OMB  directive  for  all  executive-department 
agencies  were  already  in  place  and  operative  within  the  DOD. 

The  first  of  the  two  DOD  directives  presented  the 
overall  policies  for  systems  acquisiton  within  the  DOD  and 
established  the  four  major-decision  milestones,  which  the 
SECDEF ,  as  agency  head,  must  consider.  These  four  major- 
decision  milestones  were:  Milestone  0 — Program  Initiation, 
Milestone  I — Demonstration  and  Validation,  Milestone  II — 
Full-Scale  Engineering  Development,  and  Milestone  III — 
Production  and  Deployment.  The  position  of  Defense  Acquisition 
Executive  (DAE)  within  the  Office  of  the  Secretary  of  Defense 
(OSD)  was  also  established  for  systems-acquisition  matters. 
Among  other  policies ,  this  first  directive  also  set  forth 
the  program  manager's  responsibilities  and  his  authority, 
the  test-and -evaluation  considerations,  and  the  requirement 
to  explore  competitive  systems-design  concepts.  Systems 
programs  involving  an  anticipated  cost  of  $75  million  in 
research,  development,  test  and  evaluation  (RDT&E)  or 
$300  million  in  production  were  identified  in  the  directive 
as  programs  to  be  considered  for  designation  as  major  systems 
acquisitions . 


79 


The  second  directive.  Major  Systems  Acquisition  Process, 
covered  several  areas,  including,  systems-acquisition  advisory 
councils,  mission-area-analysis  requirements,  scheduled 
program  reviews  and  Secretary  of  Defense  (SECDEF)  decision 
making,  program  management  considerations,  major  systems- 
program  documentation,  and  program  issues  to  be  addressed  at 
each  of  the  major-decision  milestones.  Provisions  were  made 
in  this  directive  for  the  establishment  of  two  advisory 
councils:  The  Defense  Systems  Acquisition  Review  Council  (DSARC) 
and  the  (Service)  Systems  Acquisition  Review  Council  ([SJSARC). 
The  DSARC  was  established  to  advise  the  SECDEF  on  the  major- 
decision  milestones  for  systems-acquisition  programs. 

The  DAE  is  the  chairman  of  the  DSARC  and  its  members 
are  senior  management  officials  from  OSD.  Prior  to  each 
major-decision  milestone,  the  DSARC  must  review  the  program 
under  consideration  and  make  recommendations  to  the  SECDEF 
on  the  program. 

The  ( S ) S ARC  functions  similarly  to  the  DSARC  but 
at  the  service-headquarters  level.  The  (S)SARC  has  senior 
management  officials  as  its  members  and  must  meet  prior  to 
each  major-deicsion  milestone  to  review  the  program  under 
consideration.  As  a  result  of  this  review,  the  (S)SARC  makes 
recommendations  on  the  program  to  the  Service  Secretary  (SERVSEC) . 

The  second  DOD  directive  also  stipulated  that  two 
major  documents  be  submitted  for  proposed  and  approved  systems 
programs  to  support  the  DSARC  and  (S)SARC  reviews  and  decisions 
of  the  Secretary  of  Defense:  the  Mission  Elements  Need 
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Statement  (MENS)  and  the  Decision  Coordinating  Paper  (DCP) . 
The  MENS  must  identify  the  mission  need  in  terms  of 
the  task  to  be  performed.  An  assessment  of  the  projected 
enemy  threat  through  the  time  frame  of  the  capability  must 
be  presented  in  the  MENS  and  the  existing  DOD  capability  to 
accomplish  the  mission  must  be  identified.  Constraints  deal¬ 
ing  with  operational,  logistical,  and  NATO  standarization 
must  be  stated  in  the  MENS,  with  an  assessment  of  the  impact 
of  not  acquiring  the  capability.  A  program  plan  to  examine 
competitive  alternative  systems  and  to  establish  a  systems 
program  office  must  also  be  presented  in  the  MENS.  Upon 
completion,  this  document  must  be  submitted  to  the  SECDEF 
by  the  component  head  for  the  Milestone  0  decision. 

The  Decision  Coordinating  Paper  must  be  prepared 
by  the  DOD  component  and  must  summarize  the  program.  Its 
principal  purpose  is  to  support  the  DSARC  and  (S)SARC  reviews 
and  the  decisions  that  the  SECDEF  must  make  at  the 
Milestone  I,  II,  and  III  major-decision  points.  The  content 
and  issues  documented  by  each  DCP  should  accurately  and  com¬ 
pletely  reflect  the  program  while  focusing  on  the  particular 
program  phase  that  is  the  subject  of  the  SECDEF ' s  upcoming 
actions.  The  DCP  must  include  an  approved  MENS,  descrip¬ 
tions  of  alternative  programs,  a  summary  of  the  acquisition 
strategy,  business-planning  information,  program-management 
plan,  areas  of  program  uncertainty,  required  resources  of 
each  alternative,  a  logistics  annex,  and  test-and-evaluation 
(T&E)  planning,  and  status  data.  The  results  of  DSARC  and 


(S)SARC  reviews  and  the  SECDEF ' s  previous  decisions  and 
directions  must  also  be  included  in  the  DCP . 

A  DELTA  chart  documenting  the  DCP-approval  cycle  is 
presented  as  Figure  7.  The  cycle  begins  with  the  SECDEF' s 
decision  that  was  documented  in  a  DCP  and  ended  the  previous 
phase.  His  decision  has  resulted  in  an  update  of  the  PPBS 
and  a  program-management  directive  for  the  systems-program 
office.  The  systems-program  office  then  carries  out  the 
next  phase  of  the  systems-acquisition  process ,  culminating 
in  the  recommendations  to  be  developed  for  the  next  phase. 
These  recommendations  are  then  given  to  the  service  head¬ 
quarters  to  develop  a  draft  DCP.  Next,  the  draft  DCP  is 
reviewed  by  the  (S)SARC  and  recommendations  made  to  the 
Service  Secretary.  The  Service  Secretary  then  takes  the 
appropriate  action  on  the  DCP  and  forwards  it  to  the  DAE 
and  DSARC ;  the  DSARC  makes  its  recommendations  to  the  SECDEF. 
Finally,  the  SECDEF  takes  action  on  the  DCP  and  issues  his 
decision,  thus  completing  the  cycle. 

DOD  directive  5000.2,  Mac  or  Systems  Acquisition 
Process,  set  forth  the  phases,  scheduled  program  reviews , 
and  SECDEF  major-decision  milestones  that  make  up  the  major 
systems-acquisition  process.  A  DELTA  Chart  representing  this 
process  is  presented  as  Figure  8.  As  depicted  by  this  chart, 
the  systems-acquisition  process  begins  with  a  continuing 
mission  analysis  by  the  service  that  can  result  in  the 
identification  of  one  or  more  mission  needs.  A  formal  MENS 


Figure  7.  Decision  Coordinating  Paper  (DCP)  Approval  Cycle 
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is  then  submitted  for  the  approval  of  Service  Headquarters, 
the  Service  Secretary,  and  the  SECDEF .  Once  the  SECDEF ' s 
approval  has  been  secured  for  program  initiation,  the  program 
moves  to  the  conceptual  phase.  When  the  work  of  the  conceptual 
phase  has  been  completed  by  the  program  office,  a  draft  DCP 
is  developed  by  the  service  headquarters.  The  DCP  is  then 
carried  through  the  DCP-approval  cycle  (Figure  7) .  The 
successive  phases  of  the  process,  demonstration  and  validation 
and  full-scale-engineering  development  then  proceed.  Each 
phase  is  followed  by  a  DCP-approval  cycle  culminating  in  a 
major-decision  milestone  by  the  SECDEF.  The  final  decision 
approves  the  system  for  production  and  deployment.  As 
production  proceeds,  more  items  are  fielded,  resulting  in  an 
initial  operational  capability  (IOC)  for  the  service. 

An  analysis  of  the  several  steps  in  each  phase  of 
the  systems-acquisition  process  is  presented  in  the  next 
section  of  this  chapter. 

The  Phases  of  the  Systens-Aaquisition  Process 

The  top-level  DOD  policies  for  systems  acquisition 
were  presented  in  the  two  DOD  directives  previously  mentioned 
(DOD,  1977a, b).  These  policies  are  implemented  within  the 
services  by  each  headquarter's  issuing  its  own  systems- 
acquisition-policy  directives.  A  brief  summary  of  these 
individual  directives  is  presented  here. 
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The  Air  Force  has  one  directive.  Acquisition  Program 
Management  (USAF,  1977),  to  implement  the  DOD  directives  and 
another  Statement  of  Operational  Need  (USAF,  1978)  to 
develop  the  operational-needs  documentation,  specifically  the 
MENS.  The  Department  of  the  Navy  has  Systems  Acquisition 
in  the  Department  of  the  Navy,  and  Systems  Acquisition  Process 
in  the  Department  of  the  Navy  (Navy,  1978a, b);  the  Army  has 
Basic  Policies  for  Systems  Acquisition  and  Systems  Acquisition 
Review  Council  Procedures  (Army,  1978  a,b) ;  and  the  Marine 
Corps  has  Systems  Acquisition  Management  Manual  (Navy,  1979) 
to  implement  the  DOD  policies. 

The  policies  of  the  four  services  outlined  in  the 
several  directives  above  are  different  in  their  approaches; 
they  vary  in  the  level  of  detail  presented  and  in  the  actual 
procedures  to  be  followed  within  each  service.  However, 
there  are  areas  common  to  each  acquisition  approach,  which 
are  discussed  in  this  section.  Each  of  the  four  major 
phases  of  the  systems  process  are  examined  here  to 
identify  those  activities  that  make  up  each  phase  and  that 
must  be  carried  out  to  comply  with  the  two  DOD  directives. 

A  general  description  of  each  activity  is  presented.^ 

The  first  phase  to  be  examined  to  determine  its  basic 
activities  is  the  one  preceding  the  program-initiation 
decision.  During  this  phase,  the  mission  area  to  be  analyzed 

3For  a  more  definitive  description  of  the  detailed  procedures 
of  the  several  phases,  see  Project  Manager’s  Guide  (Navy,  1977). 
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is  decided  and  the  MENS  is  developed.  DOD  Directive  5000.2 
(Major  Systems  .Acquisition  Process)  set  forth  the  purpose 
of  the  MENS.  These  requirements  were  amplified  by  the  Under 
Secretary  of  Defense,  Research  and  Engineering,  in  his 
memorandum  Mission  Element  Seed.  Statement  (Defense,  1978)  . 

This  memorandum  outlined  the  MENS  and  explicitly  established 
its  role  in  the  systems-acquisition  process. 

The  development  of  a  MENS  depends  on  three  activities 
that  must  be  carried  out  on  a  continuing  basis:  First, 
that  area  of  the  Defense  mission  that  the  service  involved 
is  responsible  for  must  be  effectively  analyzed.  Second, 
the  service  must  be  cognizant  of  the  existing  and  planned 
capabilities  that  could  be  used  to  fulfill  a  particular 
mission-element  need.  Third,  intelligence  agencies  must 
carry  out  the  threat  assessments  related  to  the  mission 
area  under  consideration. 

The  continuous  performance  of  these  three  activities 
is  critical  to  the  accomplishment  of  the  mission-area-analysis 
phase  of  a  systems-acquisition  process.  Considering  these 
activities  as  the  foundation  for  the  phase,  the  activities, 
or  the  steps,  necessary  to  carry  out  the  analysis  for  the  phase 
and  their  normal  sequence  of  accomplishment  are  as  follows: 

1.  Define  the  Mission  Seed  by  identifying  the 


problem  and  the  constraints  and  relating  the  desired  need  to 
other  needs  and  to  existing  or  projected  programs. 


2.  Identify  Mission  Need  in  Terms  of  Mission-Element 
Tasks  by  establishing  the  mission  need  in  a  hierarchy  of 
goals  or  objectives  for  the  mission  area.  The  top-most  goals 
would  be  heavily  value  laden  while  the  lower  level  goals 
would  be  more  concrete,  objective,  and  quantifiable. 

3.  Identify  Known-Solution  Candidates  by  identifying 
existing  and  planned  capabilities  that  could  be  used  to 
fulfill  the  mission-element  need.  Not  acquiring  any  capability 
is  clearly  a  viable  option  at  this  point. 

4.  Develop  Enemy-Threat  Scenarios  by  modelling 
the  threat  over  the  time  for  which  a  capability  is  required 
by  quantifying  the  numbers  and  performance  characteristics 
of  the  threat. 

5.  Assess  the  Impact  of  Not  Acquiring  the  Capability 

to  determine  if  a  recommendation  should  be  made  to  approve 
the  MENS  and  thereby  initiate  a  program.  If  the  need  is 
not  acquired,  then  the  existing  capability  must  be  evaluated 
against  the  threat;  if  the  need  is  acquired,  then  its  impact 
on  manpower,  training  facilities,  force  size,  and  NATO 
considerations  must  be  evaluated.  This  overall  effort  should 
culminate  with  the  development  of  the  MENS  by  service  head¬ 
quarters. 

6.  Staff  and  Process  MENS  (Milestone  0,  MENS 
Approval  Cycle)  through  the  several  headquarters  and  to  the 
SECDEF  for  his  possible  approval.  This  is  the  program- 
initiation  decision. 
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7.  Implement  the  Conceptual  Phase  by  establishing 
a  plan  to  carry  out  the  next  phase,  if  the  MENS  is  approved 
by  the  SECDEF;  identify  the  overall  resources  and  the  schedule 
to  meet  Milestone  I.  A  systems-program  office  should  be 
included  in  the  planning. 

The  several  activities  for  the  mission-area-analysis 
phase  have  been  incorporated  in  the  DELTA  chart  of  Figure  9. 
This  chart  includes  a  feedback  loop  for  the  phase  that 
illustrates  the  iterative  nature  of  the  analysis  effort. 
Personnel  from  several  different  disciplines  are  required  to 
carry  out  the  activities  of  this  phase.  These  personnel 
could  include  mission-oriented  operational  personnel, 
operations  analysts,  intelligence  analyists,  engineers, 
research  and  development  specialists,  manpower  planners,  and 
logistics  experts. 

The  second  phase  for  which  the  basic  activities  are 
determined  is  the  conceptual  phase.  From  the  decision-making 
standpoints  of  the  service  headquarters  and  the  OSD,  the 
principal  physical  output  of  this  phase  is  the  DCP  and  its 
associated  support  documents.  Although  the  DCP  has  no 
intrinsic  value  of  its  own,  it  does  motivate,  justify,  and 
establish  a  particular  program.  DOD  Directive  5000.2 
(Defense,  1977b)  set  forth  the  purpose  of  the  DCP,  indicated 
what  it  should  accomplish,  and  established  the  issues  that 
should  be  addressed  at  each  major-decision  milestone. 
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Figure  9.  The  Activities  of  the  Mission-Area- 
Analysis  Phase  of  the  Systems-Acquisition  Process. 
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The  three  activities  needed  to  carry  out  the  conceptual 
phase  are  the  service's  continuing  mission-area  analysis, 
the  establishment  of  the  implementation  plan  from  the  previous 
phase,  and  the  development  of  the  technology  base  by  industry, 
government  laboratories,  and  academic  institutions.  The 
activities  or  steps  required  to  carry  out  the  analysis  of 
the  conceptual  phase  and  their  normal  sequence  of  accomplish¬ 
ment  are: 

1.  Define  the  Acquisition  Problem  by  identifying 
program  constraints  of  all  types  and  the  resources  limitations, 
including  funding,  manpower,  and  facilities ,  and  by  determining 
the  interactions  with  current  or  projected  programs.  The 
relationship  of  the  mission-element  need  to  the  threat  and 

any  other  considerations  should  be  reaffirmed. 

2.  Identify  Program  Goals  and  Objectives  by  tracing 
them  to  the  mission  element  need  and  by  viewing  this  need  in 

a  hierarchy  of  goals  associated  with  the  overall  mission  area. 

3.  Identify  Alternative  Candidate  Systems 

by  determining  acceptable  candidate  solutions.  Alter¬ 
natives  should  be  competitive  yet  responsive  to  the  mission- 
element  need.  Proposals  for  candidate  solutions  could  be 
received  from  industry,  academic  institutions,  or  government 
laboratories . 

4.  Develop  Models  through  evaluating  operational 
considerations  and  acquisition  approaches.  A  wide  variety 
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of  model  types  should  be  considered  for  use.  Risk  factors 
should  be  considered  in  developing  the  models. 

5.  Evaluate  Alternative  Candidate  Systems 

by  using  the  models  to  carry  out  cost-performance  tradeoffs 
to  identify  one  or  more  alternative  systems  for 
consideration  in  the  demonstration  and  validation  phase. 

6.  Proceed  or  Not  to  the  Validation  and  Demonstration 
Phase  (Milestone  I  Decision  Point)  by  carrying  out  DCP  approval 
cycle  (as  outlined  in  Figure  7) . 

7.  Implement  the  Demonstration  and  Validation  Phase 
by  establishing  a  plan  necessary  to  carry  out  the  next  phase. 

For  example,  the  test  program  and  the  acquisition  strategy 
must  be  carefully  planned  and  coordinated. 

The  several  activities  for  the  conceptual  phase  are 
incorporated  in  the  DELTA  chart  of  Figure  10,  which  includes 
a  feedback  loop  to  indicate  the  iterative  nature  of  the 
analysis  effort.  Many  disciplines  are  necessary  for  this 
phase  and  could  include  the  following  personnel:  engineers, 
mission-oriented  operational  personnel,  logistic-oriented 
personnel,  contract  lawyers,  businessmen,  management  scientists, 
operations  analysts  and  intelligence  personnel,  and 
financial  management  personnel. 

The  third  phase  for  which  basic  activities  must  be 
identified  is  the  demonstration-and -validation  phase.  Again 
from  the  top-level  decision-making  standpoint,  the  physical 
output  of  this  phase  is  the  DCP  and  its  supporting  documents. 
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Figure  10.  The  Activities  of  the  Conceptual  Phase 
of  the  Systems-Acquisition  Process. 
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As  noted  above,  the  DCP ' s  true  value  lies  in  its  capability 
to  effectively  present  the  system  and  its  status.  The  mission- 
area  analysis  continues  in  this  phase  but  is  primarily 
used  to  validate  the  identified  mission-element  need.  The 
activities  of  this  phase  are  similar  to  those  of  the  concep¬ 
tual  phase  but  are  carried  out  in  greater  depth  and  more 
detail.  The  planning  from  the  previous  phase  (step  7)  is 
used  to  implement  this  phase. 

The  activities  that  are  required  to  carry  out  the 
analysis  of  the  demonstration  and  validation  phase  and  their 
normal  sequence  of  accomplishments  are: 

1.  Define  the  Systems  Problem  by  reviewing  the 
problem  in  light  of  the  further  insight  gained  during  the 
conceptual  phase  and  by  examining,  in  depth,  the  operational 
and  technical  aspects  of  the  problem.  Constraints,  inter¬ 
actions,  and  alterable  components  should  be  reviewed  and 
then  validated  or  revised. 

2.  Examine  and  Validate  Program  Goals  and  Objectives 

by  tracing  them  to  the  mission-element  need.  More  concrete 
and  objective  goals  should  be  established  as  the  program 
proceeds  through  the  acquisition  process. 

3.  Validate  and  Refine  Systems  or  Program  Alternatives 
by  analyzing  the  alternatives  from  the  conceptual  phase  to 
determine  critical  systems-ef f ectiveness  parameters.  This 

is  usually  done  by  contract  with  an  industry  or  government 

\ 


laboratory.  Major  characteristics  (technique,  cost,  and 
schedule)  of  the  alternatives  become  more  readily  identifiable 

4.  Develop  Models  for  evaluating  systems  or  program 
alternatives;  i.e.,  procure  and  test  an  advanced  development 
model  (ADM).  All  types  of  models,  such  as,  simulation,  bread¬ 
board,  and  mathematical  system-reliability  models,  may  be  used 
in  this  step. 

5.  Evaluate  System  or  Program  Alternatives  by  using 
the  models  from  the  previous  step  (step  4)  to  evaluate  the 
alternatives  so  that  the  relationship  between  systems  perfor¬ 
mance  and  technical  capabilities  can  be  optimized  while 
considering  the  constraints,  such  as,  force-structure  and 
funding.  The  goal  of  this  phase  is  to  recommend  the  most 
desirable  systems  design  for  full-scale  development.  The 
results  of  development,  test,  and  evaluation  ( DT&E)  carried 
on  during  this  phase  provide  a  significant  input  to  this 
process . 

6.  Proceed  to  Full-Scale  Development  (Milestone  II 
Decision  Point)  by  carrying  out  the  DCP  approval  cycle  as 
outlined  in  Figure  7. 

7.  Implement  the  Full-Scale  Development  Phase  by 

establishing  the  plan  necessary  to  carry  out  the  next 
phase.  For  example,  the  test  program  and  the  acquisition 
strategy  must  be  carefully  planned. 

The  several  activities  for  the  demonstration-and- 
validation  phase  are  incorporated  in  the  DELTA  chart  of 
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Figure  11,  which  includes  a  feedback  loop  to  indicate  the 
iterative  nature  of  the  analysis  effort.  Again,  personnel 
from  many  disciplines  should  be  employed  in  this  phase. 

The  final  phase  to  be  examined  is  that  of  full-scale 
development.  As  before,  the  end  product  of  the  phase 
will  be  the  DCP  and  its  supporting  documents  presenting 
the  program  or  system.  Certainly  the  full-scale-engineering 
test  model  is  important  but  the  emphasis  here  is  on  the 
decision  process  and  the  DCP ' s  relation  to  it.  Mission-area 
analysis  continues  in  this  phase  as  a  means  to  validate  the 
mission  need.  The  planning  from  the  previous  phase  (step  7) 
is  used  to  implement  this  phase. 

The  following  activities  to  be  carried  out  by  the 
service  for  this  phase  are  presented  in  their  normal 
sequence : 

1.  Define  the  "Production  Problem,  in  depth,  through 
a  detailed  engineering  analysis,  including  the  technical 
and  operational  constraints  and  all  facets  of  such  factors 

as  the  NATO  standardization  and  interoperability  requirements. 

2.  Examine  and  Validate  Program  Goals  and  Objectives 
by  tracing  the  top-level  goals  through  the  system  design  to 
the  actual  system  components  considering  the  mission  area 

and  the  threat.  Each  system  component  should  contribute 
upward  (through  subsystems  and  the  system  itself)  to  the 
stated  mission  need. 
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Figure  11.  The  Activities  of  the  Demonstration-and 
Validation  Phase  of  the  Systems-Acquisition  Process. 
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3.  Validate  the  Projected  Alternative  Systems  to 

insure  that  the  cost,  schedule,  and  performance  goals  are 
being  met  by  the  system  design.  This  validation  relates 
primarily  to  the  system  itself  and  not  to  the  mission  need. 

4.  Develop  Models  for  evaluating  the  projected 
system  by  relating  it  to  the  mission  need.  The  characteristics 
of  the  production  item  should  be  compared  to  the  mission 

need  in  a  constrained  environment,  such  as,  cost  restrictions, 
personnel,  and  maintenance,  and  training  requirement  difficul¬ 
ties.  As  before,  all  types  of  models  may  be  used  in  this 
step . 

5.  Evaluate  the  System  by  using  the  models  developed 
in  the  previous  steps  (step  4)  to  test  and  evaluate  the  system 
sc  that  a  substantiated  go/no-go  recommendation  may  be  made 
for  the  production  of  the  system.  Initial  operational  test 
and  evaluation  (IOT&E)  of  the  system  itself  is  most  commonly 
used. 

6.  Proceed  to  the  Production  and  Deployment  Phases 

(Milestone  III  Decision  Point)  by  carrying  out  the  DCP 
approval  cycle  as  outlined  in  Figure  7. 

7.  Implement  Production  and  Deployment  by  establish¬ 
ing  the  necessary  plans  for  carrying  out  the  next  two  phases. 
Parts  of  this  planning  phase  will  have  begun  as  early  as  the 
conceptual  phase, and  production  is  heavily  dependent  on  the 
planning  of  the  previous  phases. 
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The  several  activities  for  the  full-scale-engineering 
development  are  incorporated  in  the  DELTA  chart  of  Figure  12, 
which  includes  a  feedback  loop  indicating  the  iterative  nature 
of  the  analysis  effort.  Although  the  same  disciplines  men¬ 
tioned  before  would  be  employed  in  this  phase,  there  would 
be  a  greater  emphasis  here  on  the  design-engineering  and 
manufacturing  disciplines.  Mission-oriented  operational 
personnel  and  personnel  from  the  intelligence  community  would 
be  necessary  during  the  program  for  the  mission-area 
analysis  continuously  conducted  during  the  acquisition 
cycle . 

A  review  of  the  DELTA  charts  (Figures  9  through  12) 
reveals  an  analysis  effort  that  is  generally  sequential  in 
nature  and  may  be  iterative  as  more  complete  and  refined 
information  is  gained  through  analysis.  The  activities 
shown  on  the  charts  can  be  affected  by  other  activities  or 
events  outside  each  analysis  cycle.  However,  the  primary 
interest  hare  is  to  show  the  general  flow  of  the  analysis 
effort  as  it  is  observed  from  the  OSD  and  service-headquarters 
level.  A  variety  of  contractor  and  government  organizations 
may  contribute  to  the  accomplishment  of  each  of  the  activities. 
But,  because  the  systems-program  office  is  the  focal  point 
for  all  program  matters,  the  term  servioe  is  normally  used  in 
each  activity  box  to  show  the  responsibility  to  accomplish 
that  activity. 
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Figure  12.  The  Activities  of  Full-Scale  Development 
Phase  of  the  Systems-Acquisition  Process. 
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In  the  next  section  the  activities  in  each  acquisition- 
process  phase  are  compared  to  the  steps  of  the  systems- 
engineering  framework. 

The  Systems-Engineering  Framework  and 
the  DOD  Systems-Aaquisition  Process 

This  section  examines  a  possible  correlation  between 
the  svstems-engineering  framework  (Section,-  p.67)  and  the 
phases  of  the  systems-acquisition  process  (Section,  p.  84) . 

The  phases  of  the  framework  and  the  steps  of  each  phase  will 
be  examined. 

The  phases  of  the  systems-engineering  framework  are 
almost  equivalent  to  the  phases  in  the  Defense  systems- 
acquisition  process.  A  review  of  Table  1  shows  that, 
with  one  exception,  there  is  a  one-to-one  correspondence 
between  the  phases  of  the  two  classification  systems;  the 
exception  is  the  phase  termed  project  planning  by  Hall  (1969). 
For  this  phase,  two  comparative  phases  may  be  identified  in 
the  Defense  systems-acquisition  process — the  conceptual  phase 
and  the  demonstration-and-validation  phase.  The  activities 
of  each  phase  (conceptual  and  demonstration  and  validation) 
are  equivalent  to  the  systems-engineering  project-planning 
phase,  except  that  the  demonstration-and-validation  phase  is 
carried  out  in  greater  depth  and  in  a  more  definitive  fashion 
than  the  conceptual  phase.  These  two  phases  can  be  considered 
separate  entities  within  the  systems-engineering  framework 
without  any  loss  of  generality.  The  time  dimension  of  the 
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TABLE  1 


COMPARISON  OF  THE  TIME-DIMENSION  PHASES  OF  THE 
SYSTEMS-ENGINEERING  MORPHOLOGY  WITH  THE  PHASES 
OF  A  SYSTEMS-ACQUISITION  PROCESS 


Systems-Engineering 
Dimension  Phases 1 

Systems-Acquisition 

Phases2 

Comments 

Program  planning 

Mission  analysis  and 
the  development  of 
a  MENS 

Concern  is  for  roles  and 
and  missions;  activi¬ 
ties  and  goals  are 
identified. 

Project  planning 

Conceptual 

Demonstration  and 
validation 

Emphasis  is  on  a  specific 
project;  identification, 
cultivation,  evaluation 
of  viable  candidate 
solutions  are  main 

concerns . 

System  development 

Full-scale 

development 

The  system  and  principal 
items  necessary  for  sup¬ 
port  are  designed,  fabri¬ 
cated,  tested;  concern 
is  for  system  components, 
not  system  alternatives. 

Production 

Production 

The  functions  performed 
are  the  same. 

Distribution 

Deployment  or 
phase-in 

Both  overlap  with  opera¬ 
tion  phase ,  but  func¬ 
tions  performed  are  the 
same. 

Operation 

1 .  Operation/support 

2 .  Maintenance/repair 

3.  Modification/ 

retrofit 

Several  activities  occur 
in  parallel  during  this 
phase . 

Retirement 

Retirement  or 
phase-out 

Both  overlap  with  the 
operation  phase. 

^ee  The  Systems-Engineering  Framework  (p.67  )  . 


2Systems-acquisition  phases  are  defined  in  Appendix  A  (p.325). 
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systems-acquisition  process  is  conveniently  and  properly 
modelled  using  the  systems-engineering  framework.  The 
comments  column  of  Table  1  further  compares  the  two  classifi¬ 
cation  systems. 

The  logic  dimension  steps  as  presented  in  The  Systems- 
Engineering  Framework  (p.  67)  have  also  been  compared  to  the 
several  activities  of  each  phase  of  the  systems-acquisition 
process  (Section,  p.  84) .  These  comparisons  are 
presented  in  Tables  2,  3,  4,  and  5  for  the  mission-analysis 
phase,  conceptual  phase,  demonstration-and-validation  phase, 
and  full-scale-development  phase,  respectively.  Each  indicates 
a  favorable  comparison  between  the  logic  dimension  steps  of 
systems  engineering  and  the  activities  of  each  systems- 
acquisition  phase.  The  logic  steps  of  the  systems-engineering 
morphology  provide  an  appropriate  approach  for  modelling 
the  activities  of  each  phase  in  the  systems-acquisition  process. 

An  overall  review  of  the  results  presented  in 
Tables  1  through  5  shows  that  the  phases  of  a  systems- 
acquisition  process  closely  align  themselves  with  the  phases 
of  the  systems-engineering  framework.  This  is  consistent 
with  findings  that  indicate  research-and-development  programs 
could  be  viewed  as  having  multistage  (or  multiphase)  aspects 
(Brandenburg  and  Langenberg,  1969;  Gear,  Lockett,  and  Pearson, 
1971;  Gear  and  Lockett,  1973,  1978).  Additionally,  Albala 
(1975) indicated  an  interrelationship  between  the  phases  of  a 
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TABLE  2 


COMPARISON  OF  THE  LOGIC-DIMENSION  STEPS  OF  THE  SYSTEMS -ENGINEERING 
MORPHOLOGY  WITH  THE  ACTIVITIES  OF  THE  MISSION-ANALYSIS  PHASE 
OF  THE  SYSTEMS-ACQUISITION  PROCESS 


Systems-Engineering 
Logic-Dimension 
Steps  1 

Mission- Analy  sis 
Activities 

Comments 

Problem  definition 

Mission-need 

identification 

An  operational  need  state¬ 
ment  is  developed;  in¬ 
cludes  deficiencies, 
constraints ,  existing  and 
planned  capabilities , 
problem  interrelationships 

Value-system  design 

Identify  mission  need 
in  terms  of  mission- 
element  tasks 

Top-level  objectives  are 
determined;  objectives 
measures  are  identified. 

Systems  synthesis 

Identify  known  solu¬ 
tion  candidates 

Candidate  policies  or  capa¬ 
bilities  are  postulated; 
includes  existing  DOD 
capability  and  no  system. 

Systems  analysis 
and  modelling 

Develop  enemy-threat 
scenarios  for  time 
frame  of  required 
capability 

Threat  scenarios  are  used 
to  model  enemy 
capabilities . 

Optimization 

Assess  impact  of  not 
acquiring  or  main¬ 
taining  capability 

Candidate  solutions  are 
considered  in  context  of 
the  threat,  projected  ob¬ 
solescence,  technological 
opportunity. 

Decision-making 

Milestone  0 

(MENS  approval  cycle) 

MENS  is  processed  through 
Requirements  Command, 
Service  Headquarters , 
SERVSEC ,  DAE,  SECDEF . 

Planning  for  action 
( implementation 
of  next  phase) 

Implementation/ 
conceptual  phase 

Plan  includes  identifying, 
exploring  competitive 
alternative  systems, 
establishing  systems- 
program  office. 

^ee  The  Systems -Engine erring  Framework  (p.  67). 
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TABLE  3 


COMPARISON  OF  THE  LOGIC-DIMENSION  STEPS  OF  SYSTE MS-ENGINEERING 
MORPHOLOGY  WITH  THE  ACTIVITIES  OF  THE  CONCEPTUAL 
PHASE  OF  THE  SYSTEMS-ACQUISITION  PROCESS 


Systems-Engineering 
Logic-Dimension 
Steps 1 

Conceptual-Phase 

Activities 

Comments 

Problem  definition 

Define  acquisition 
problem 

Missions-element  task  is 
reaffirmed  in  enemy- threat 
context;  program  con¬ 
straints  are  identified. 

Value-system  design 

Identify  program  goals 
and  objectives 

Goals  are  traceable  to 
mission-element  need. 

Systems  synthesis 

Identify  candidate- 
system  alternatives 

Candidates  satisfy  mission- 
element  needs ,  reflect 
technology  base,  and  pro¬ 
vide  acceptable  environ¬ 
ment;  all  alternatives 
are  considered. 

Systems  analysis 
and  modelling 

Develop  models  to 

evaluate  operational 
consideration  and 
acquisition 
approaches 

Cost-performance  tradeoffs 
and  logistics  are  consi¬ 
dered;  capability  to 
handle  risk  factors  is 
incorporated. 

Optimization 

Evaluate  candidate- 
system  alternatives 

Models  developed  in  the 
previous  step  are  used. 

Decision  making 

Milestone  I 

(DCP  approval  cycle) 

DCP  is  processed  through 
(S)SARC,  SERVSEC ,  DAE, 
DSARC ,  SECDEF . 

Planning  for  action 
( implementation 
of  next  phase) 

Implementation  of 
demonstration-and- 
validation  phase 

Plan  includes  test  program 
for  next  phase,  acquisi¬ 
tion  strategy,  business 
planning  to  support 
acquisition  strategy. 

^ee  The  Systems -Engineering  Framework  (p.  67) . 
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TABLE  4 


COMPARISON  OF  THE  LOGIC-DIMENSION  STEPS  OF  SYSTEMS-ENGINEERING  MORPHOLOGY 
WITH  THE  ACTIVITIES  OF  THE  DEMONSTRATION-AND- VALIDATION 
PHASE  OF  THE  SYSTEMS-ACQUISITION  PROCESS 


Systems-Engineering 
Logic- Dimension 
Steps 1 

Demons  trati on- and- 
Validation-Phase 
Activities 

Comments 

Problem  definition 

Define  systems  problem 

Technical  approaches  from 
conceptual  phase  are  ana¬ 
lyzed.  Constraints  and 
needs  are  identified  in 
context  of  operational  and 
technical  considerations . 

Value-system  design 

Examine  and  validate 
program  goals  and 
objectives 

Goals  are  traceable  to 
mission-element  need  and 
extend  to  quantifiable 
levels;  technical  consid¬ 
erations  cure  included. 

System  synthesis 

Validate  and  refine 

Technical  approaches  from 
conceptual  phase  are  vali¬ 
dated;  system  is  redefined 
and  critical  parameters 
identified. 

Systems  analysis 
and  modelling 

Develop  models  to 
evaluate  system  or 
program  alternatives 

Models  developed  to 
evaluate  system 
effectiveness . 

Optimization 

Evaluate  system  or 
program  alternatives 

Analysis,  simulation,  or 
advance  development  deter¬ 
mine  optimal  relationship 
between  operational  re¬ 
quirements,  technical 
capabilities ;  DT&E  results 
are  considered. 

Decision  making 

Milestone  II 

(DCP  approval  cycle) 

DCP  is  processed  through 
(S)SARC,  SERVSEC ,  DAE, 

DSARC ,  SECDEF . 

Planning  for  action 
( implementation 
of  next  phase) 

Implementation  of 
full-scale- 
development  phase 

Plan  includes  test  program 
for  next  phase,  updated 
acquisition  strategy, 
business  plan  to  support 
strategy. 

'see  The  Systems -Engineering  Framework  (p. 67)  • 
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TABLE  5 

COMPARISON  OF  THE  LOGIC-DIMENSION  STEPS  OF  SYSTEMS- ENG INEERING  MORPHOLOGY 
WITH  THE  ACTIVITIES  OF  THE  FULL-SCALE-DEVELOPMENT 
PHASE  OF  THE  SYSTEMS-ACQUISITION  PROCESS 


Systems-Engineering  Full-Scale 

Logic-Dimension  Development-phase  Comments 

Steps 1  Activities 

Problem  definition  Define  production  System-production  problem  is  ana- 

problem  lyzed  in  depth;  operational 

constraints  are  reviewed;  cost, 
schedule,  risk  identified;  NATO 
and  interoperability  require¬ 
ments  considered. 

Examine  and  validate  Mission-element  need  is  re¬ 
program  goals  and  affirmed  in  context  of 

objectives  threat;  top-level  goals  are 

traceable  to  subsystems, 
functions,  specifications, 
system  components . 

System  synthesis  Validate  projected  System  undergoing  full-scale 

system  alternative  development  is  validated; 

cost,  schedule,  performance 
are  insured. 

Develop  models  to  Cost  models  for  LCC  estimates ; 

evaluate  projected  simulation,  mathematical 

system  .  models  for  maintainability, 

reliability,  availability 
considerations;  IOT&E  for 
the  system. 

Optimization  Evaluate  system  Above  models  evaluate  system; 

IOT&E  results  insure  system 
satisfies  mission-element 
need;  system  is  confirmed 
cost-effective. 

Decision  making  Milestone  III  DCP  is  processed  through 

(S)SARC,  SERVSEC,  DAE,  DSARC, 
SECDEF . 

Implementation  of  Plan  includes  updated  acquisi- 

production  and  tion  strategy,  business  plan 

deployment  phase  to  support  strategy;  produc¬ 

tion  plan  for  facilities , 
quality  control,  second  source. 

^ee  The  Systems  Engineering  Framework  (p.  67  ) . 


Planning  for  action 
(implementation 
of  next  phase) 


Systems  analysis 
and  modelling 


Value-system 

design 
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development  program  and  the  major  decisions  associated 
with  the  program.  This  is  clearly  the  case  in  the  Defense 
systems-acquisition  process,  as  indicated  by  the  results 
presented  in  Table  2  through  5. 

The  knowledge  aspects  of  the  systems-engineering 
approach  are  readily  identifiable  in  the  systems-acquisition 
process.  As  pointed  out  in  The  Phases  of  the  Systems- 
Acquisition  Process  (p.84),  the  Defense  development  programs 
require  personnel  from  many  disciplines  to  effectively 
carry  them  out . 


Summary  and  Conclusions 

This  chapter  has  analyzed  the  structure  of  the 
Defense  systems-acquisition  problems.  Initially,  the 
systems-engineering  framework  was  reviewed;  the  significant 
aspects  of  the  DOD  systems-acquisition  policy  and  the  major- 
decision  milestones  related  to  the  DSARC  process  were  then 
covered.  The  various  activities  and  steps  that  make  up  the 
several  phases  of  the  systems-acquisition  process  were 
analyzed.  Finally,  the  steps,  phases,  and  knowledge  dimen¬ 
sions  of  the  systems-engineering  framework  were  respectively 
compared  to  the  phases,  activities,  and  discipline  require¬ 
ments  of  the  systems-acquisition  process  to  determine  if 
there  is  a  pairwise  correlation  between  the  elements  of  the 


three  sets. 


Based  on  the  analysis  in  this  chapter  and,  in 


particular,  on  the  correlations  identified  in  The  Systems- 
Engineering  Framework  and  the  DOD  Systems- Acquisition  Process 
(pj.00)  ,  the  following  conclusions  are  drawn: 

1.  That  the  systems-acquisition  process  is  multi¬ 
discipline  in  nature  and  is  representative  of  systems- 
engineering  problems  in  its  knowledge-requirements  dimension. 

2.  That  the  time  dimension  of  the  systems-acquisition 
process,  represented  by  its  phases,  generally  approximates 
the  time  dimension  and  phases  of  the  systems-engineering 
framework. 

3.  That  the  logic  dimension  of  the  phases  of  the 
systems-acquisition  process  include  a  sequence  of  activities 
(or  steps)  that  are  comparable  to  the  steps  (or  activities) 
of  the  logic  dimension  of  the  systems-engineering  framework. 

4.  That,  given  conclusions  1  through  3  above,  the 
systems-acquisition  process  is  the  systems-engineering  problem 
that  can  be  carried  out  within  the  conceptual  framework  of 

the  systems-engineering  morphological  box.  Furthermore,  the 
tools  and  techniques  of  systems  engineering  can  be  applied  to 
the  systems-acquisition  process  in  accordance  with  currently 
accepted  practice  (see  Hall,  1962;  Chestnut,  1966  and  1967; 
Warfield,  1976;  Sage,  1977a,  1977b;  Gibson,  1977a,  1977b, 

1979)  . 

Therefore  the  systems-engineering  morphological  box  is 
used  to  model  the  systems-acquisition  process  throughout 
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the  remainder  of  this  paper.  Figure  13  presents  the  three- 
dimensional  framework  for  systems  engineering  that  has  been 
revised  to  incorporate  those  aspects  of  the  systems-acquisition 
process  previously  identified  as  being  peculiar  to  that  process. 

Throughout  the  course  of  this  research  a  number  of 
DOD  personnel  were  interviewed  (see  Appendix  B,  p.334)  in 
relation  to  the  systems-acquisition  process.  Additionally, 
a  great  number  of  DOD  directives  and  other  documents  were 
reviewed . 

Based  on  these  sources,  it  is  the  author's  perception 
that  the  DOD  personnel  currently  involved  in  the  system- 
acquisition  process  often  do  not  view  the  process  in  the 
context  of  the  systems-engineering  framework.  Many  do  not 
see  the  overall  comparison  between  a  systems-acquisition 
process  and  the  systems-engineering  framework.  They  do  not 
perceive  the  parallelism  between  the  several  phases  of  the 
systems-acquisition  process  with  their  comparable  activities 
and  decision  points.  This  is  particularly  true  of  the 
mission-analysis  (program-planning)  phase  of  the  systems- 
acquisition  process. 


Business 


Mi  ii tary /Naval 
Science 


Mission-Analysis  Phase 


£  Conceptual  Phase 

Denonstrati on-and-Validati on  Phase 


Ficrure  13.  The  Morphological  Box  for  Systems  Engi 
AdaDted  for  Systems  Acauisitions  (after  Hall,  1969 
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Chapter  4 


THE  DEVELOPMENT  OF  THE  SYSTEMS -ENGINEERING  METHODOLOGY 
FOR  APPLICATION  TO  THE  SYSTEMS-ACQUISITION  PROCESS 
OF  THE  DEPARTMENT  OF  DEFENSE 

Introduction 

During  the  past  several  years,  the  procurement  policies 
of  the  federal  government  have  come  under  close  scrutiny  from 
many  sources.  In  the  early  1970 's,  the  Commission  on  Govern¬ 
ment  Procurement  studied  these  policies  and  issued  a  report 
that  documented  a  number  of  recommendations.  These  recommenda¬ 
tions  evolved  into  the  revised  procurement  policies  presented 
in  the  OMB  circular,  Major  Systems  Acquisition ,  A-109  (0MB, 
1976a) ,  issued  by  the  Office  of  Federal  Procurement  Policy 
(OFPP)  on  April  5,  1976. 

This  circular  defined  a  major  system  as,  "that  com¬ 
bination  of  elements  that  will  function  together  to  produce 
the  capabilities  required  to  fulfill  a  mission  need."  Major 
programs  were  defined  as  those  that  "are  directed  at  and 
critical  to  fulfilling  an  agency  mission,  entail  the  alloca¬ 
tion  of  large  resources,  and  warrant  special  management 
attention."  OMB  Circular  A-109  also  established  key  systems- 
acquisitions  policies  by  requiring  that: 

1.  Program  management  be  strengthened.  A  program 
manager  must  be  designated  with  complete  authority  and 
responsibility  for  the  systems-acquisition  effort  as  soon 
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as  a  need  has  been  approved  by  the  agency  head.  A  program 
manager  must  understand  the  user's  needs  and  constraints,  be 
familiar  with  development  principles,  and  possess  the  requisite 
management  skills  and  experience. 

2.  Congress  be  -informed  of  a  new  need  process.  As 
soon  as  a  new  mission  need  has  been  approved  by  an  agency  head 
and  authorization  has  been  given  to  explore  alternatives, 
Congress  should  be  informed.  Congress  must  be  able  to  consider 
a  need  on  its  merits  early  in  the  acquisition  process  before 
major  resources  have  been  committed  and  a  systems  problem 

has  been  resolved.  This  early  notification  enables  Congress  to 
continuously  compare  the  needs  of  all  agencies  and  their 
associated  acquisition  programs. 

3.  An  acquisition  executive  be  designated.  "The  head 
of  each  agency  that  acquires  major  systems  will  designate 

an  executive  to  integrate  and  unify  the  management  process 
for  the  agency's  major  systems  acquisitions  and  to  monitor 
implementation  of  the  policies  and  practices  set  forth  in  the 
Circular"  (OMB,  1976a) . 

4.  A  need  be  expressed  in  mission  terms.  An  agency's 
needs  are  the  capabilities  required  to  successfully  carry  out 
its  overall  mission.  The  need  for  a  new  major  system  must  be 
expressed  in  mission  terms,  not  defined  in  equipment  terms,  so 
that  Congress  and  the  agency  head  can  meaningfully  trade-off 
needs  after  considering  the  overall  capabilities,  resources, 
and  priorties. 
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5.  Alternative  systems  he  explored.  Alternative 
systems-design  concepts  must  be  explored  within  the  context  of 
the  agency's  mission  need  and  program  objectives  and 
innovative  and  conceptual  competition  must  be  actively 
generated  from  industry.  Alternatives  should  be  formally 
solicited  without  bias  from  industry  and  other  sources  based 
on  mission  and  functional  specifications.  The  best  possible 
design  should  be  sought  by  thoroughly  exploring  alternative 
systems-design  concepts  and  tradeoffs  of  capability  for 
schedule  or  cost. 

6.  The  four  major  program  decisions  be  reserved  for 
the  agency  head.  The  right  to  make  four  major  milestone 
decisions  must  be  reserved  for  the  agency  head.  These  mile¬ 
stones  are  the  go/no-go  decisions  that  require  the  reaffirma¬ 
tion  of  a  prior  decision  and  determine  whether  or  not  the 
system  development  will  proceed  into  the  next  phase.  By 
making  these  key  decisions,  the  agency  head  controls  the 
various  systems  acquisitions  within  the  agency  while  con¬ 
sidering  the  cost  and  other  limitations  of  the  agency's 
capabilities  and  resources.  The  four  key  decisions  that  the 
agency  head  must  make  are:  (1)  approval  of  a  newly  defined 
need,  (2)  selection  of  competitive  design  concepts  to  be 
advanced  to  the  test  and  demonstration  phase,  (3)  commitment 
of  a  system  to  full-scale  development,  and  (4)  commitment 

of  the  system  to  full  production. 


118 


The  overall  systems-acquisition  process  as  envisioned 
by  the  OMB  circular  was  detailed  in  the  OFPP ' s  Pamphlet  No.  1 
(0MB,  1976b) .  The  process  begins  on  the  left  of  the  DELTA 
chart  of  Figure  14  with  a  continuing  mission-area  analysis  by 
the  agency.  When  this  analysis  identifies  a  deficiency  in  an 
existing  agency  capability  or  an  opportunity  is  identified 
for  a  technological  improvement  that  will  establish  a  new 
capability,  the  deficiency  is  formally  documented  in  a  mission- 
need  statement.  This  mission-need  statement  fully  defines 
the  purpose,  projected  capability,  agency  components, 
constraints,  value,  and  priority  of  the  need.  As  shown  in 
the  DELTA  chart,  the  mission-need  statement  is  then  submitted 
to  the  agency  head  for  approval.  This  action  represents 
Milestone  0  of  the  process. 

The  approval  of  a  mission  need  starts  the  major  systems- 
acquisition  process  by  authorizing  the  agency  to  explore 
alternative-design  concepts.  A  program  manager  is  then 
designated  who,  with  a  staff,  develops  an  acquisition  strategy. 
The  acquisition  strategy  is  an  overall  approach  or  plan  to 
meet  the  program  objective  in  an  economic,  effective,  and 
efficient  manner.  Solicitations  are  then  sent  to  the 
appropriate  industry  requesting  proposals  for  alternative 
systems  designed  to  satisfy  the  approved  mission  need.  After 
proposals  are  evaluated,  parallel  short-term  contracts  are 
awarded  to  further  explore  and  define  the  alternatives.  From 
these  alternatives,  selected  concepts  are  recommended  for 


Figure  14.  A-109  (OMB,  1976),  Major  Systems-Acquisition  Process.  This 

chart  represents  an  overview  of  the  systems-acquisition  process  assuming  approval 
of  the  program  at  the  decision  points  and  a  normal  progression  towards  productio 
Each  phase  between  the  decisions  could  have  a  feedback  loop  to  provide  for  itera 
tion  of  the  analysis  effort  (see  Figs.  9,  10,  11,  12). 
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demonstration.  The  consideration  of  these  alternative 
concepts  and  the  approval  by  the  agency  head  to  enter  the 
competitive-demonstration  phase  of  the  acquisition  process  is 
Milestone  I  (Figure  14)  . 

Competitive-demonstration  contracts  are  then  awarded. 
These  demonstrations  should  verify  that  the  selected  design 
concepts  are  sound,  assure  that  they  perform  in  an 
operational  environment,  and  provide  sufficient  data  to 
evaluate  and  select  the  systems-design  concepts  to  be  continued 
into  full-scale  development.  The  decision  by  the  agency  head 
to  proceed  to  full-scale  development  is  Milestone  II  on 
the  DELTA  chart. 

Full-scale  development  is  then  carried  out.  Following 
satisfactory  test-and-evaluation  results  and  reconfirmation 
of  the  mission  need  and  program  objectives,  the  agency  head 
can  authorize  full  production.  This  is  Milestone  III 
(Figure  14) . 

The  broken  lines  of  Figure  14  represent  the  progressive 
narrowing  of  viable  alternatives  (as  a  function  of  development 
time)  to  be  chosen  as  the  solution  for  an  established  need. 
These  broken  lines  also  illustrate  the  progressive  minimiza¬ 
tion  of  participating  contractors  throughout  the  acquisition 
process . 

The  Department  of  Defense  (DOD)  has  directed  the 
implementation  of  the  policies  in  Major  Systems  Acquisition 
(0MB,  1976a)  through  two  revised  directives.  Major  Systems 
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Acquisitions  (Defense,  1977a)  and  Major  Systems  Acquisition 
Process  (Defense,  1977b).  The  provisions  of  the  first 
DOD  directive  5000.1,  defined  a  major  systems-acquisition 
program  as  one  with  projected  research,  development,  test, 
and  evaluation  (RDT&E)  cost  of  $75  million  or  a  projected 
procurement  cost  of  $300  million. 

This  directive  also  established  the  position  of 
Defense  Acquisition  Executive  (DAE)  (whose  responsibilities 
are  outlined  in  a  further  directive  [Defense,  1976b]  as  the 
focal  point  in  the  Office  of  the  Secretary  of  Defense  (OSD) 
for  systems  acquisition.  In  keeping  with  the  policies  of 
the  original  OMB  directive,  this  DOD  directive  reserved  the 
following  four  key  major-program  decisions  for  the  Secretary 


Defense 

( SECDEF) : 

1. 

Milestone 

0 

-  Decision  for  Program  Initiation 
(Approval  of  Need) 

2. 

Milestone 

I 

-  Decision  to  Proceed  to  Demonstra¬ 
tion  and  Validation 

3. 

Milestone 

•II 

-  Decision  to  Proceed  to  Full-Scale 
Development 

4. 

Milestone 

III 

-  Decision  for  Production  and 
Deployment 

Generally,  the  acquisition  policies  set  forth  in 
this  first  DOD  directive,  5000.1,  were  consistent  with  those 
already  mentioned  in  the  OMB  circular,  A-109.  However, 

5000.1  also  provided  additional  guidance  on  establishing 
advisory  councils  to  make  recommendations  in  the  systems- 
acquisition  decision  process  and  on  documenting  a  major  systems 


program. 
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The  establishment  of  a  Defense  Systems  Acquisition 
Review  Council  (DSARC)  and  (Service)  System  Acquisition  Review 
Council  ( [ S ] S ARC )  was  stipulated  in  5000.1.  The  DSARC  was 
established  to  review  major  programs  and  make  recommendations 
to  SECDEF  prior  to  the  major  decisions  at  Milestones  I,  II, 
and  III.  The  (S)SARC  was  to  perform  the  same  type  of  function 
for  the  Service  Secretary  (SERVSEC)  for  the  major-decision 
Milestones  I,  II,  and  III  before  the  DSARC,  DAE,  and  SECDEF 
considered  the  program. 

The  Gantt  chart  of  Figure  15  graphically  portrays 
the  sequence  of  the  systems-acquisition  process.  The  phases  of 
mission  analysis,  conceptual  (development),  demonstration 
and  validation,  full-scale  development,  and  production  and 
deployment  are  shown  interspaced  with  the  four  major-decision 
milestones.  The  chart  also  shows  the  sequence  of  the  (S)SARC 
review,  SERVSEC  decision,  and  DSARC  review  prior  to  the 
SECDEF  decisions  at  Milestones  I,  II,  and  III. 

A  Mission  Element  Need  Statement  (MENS)  and  a 
Decision  Coordination  Paper  (DCP)  for  each  major  program  are 
also  required  by  5000.1.  The  MENS  documents  the  mission  need 
in  terms  of  the  mission-element  tasks,  the  enemy  threat,  the 
existing  DOD  capability  to  accomplish  a  mission,  and  the 
constraints  on  a  problem.  It  also  establishes  a  program 
plan  to  identify  and  explore  competitive  alternative 
systems . 


15.  The  Sequence  of  Systems-Acquisition  Phases  and  Decision  Points 
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The  DCP,  developed  and  updated  prior  to  Milestones 
I,  II,  and  III,  consists  of  an  updated  MENS,  an  acquisition 
strategy,  a  business  plan,  a  management  plan,  the  areas  of 
program  uncertainty,  the  required  resources,  a  testing  and 
evaluation  (T&E)  plan  and  status,  the  program  issues,  DSARC 
and  (S)SARC  recommendations,  and  SECDEF  decisions  and  direc¬ 
tion.  The  DCP,  as  a  decision  vehicle  for  the  DSARC/ ( S) SARC , 
provides  the  justification  and  motivation  for  and  the  current 
development  status  of  a  program.  Directive  5000.2,  Major 
Systems  Acquisition  Process  (Defense,  1977b),  outlined  in  more 
detail  the  data  required  for  the  MENS  and  for  the  DCP  at 
Milestones  I,  II,  and  III. 

The  several  directives  discussed  above  form  a  systems- 
acquisition-policy  framework  for  the  DOD.  This  chapter 
develops  a  systems-engineering  methodology  for  systems 
acquisition  within  the  DOD  policy  framework  using  the  decision 
processes  leading  to  the  four  major-decision  milestones. 
Developing  and  processing  the  MENS  and  DCP  through  the  (S)SARC, 
SERVSEC ,  DSARC,  and  SECDEF  are  emphasized. 

The  employment  of  a  systems-engineering  approach  for 
F.&D  projects  was  discussed  by  Gibson  (1979)  .  The  need  for 
a  meaningful  project  analysis  to  evaluate  alternatives  for 
a  development  program  was  outlined  by  Werber  (1973).  Moore 
(1975)  argued  that  both  the  costs  and  benefits  of  projects 
must  be  quantitatively  considered  when  designing  an  overall 
research  program.  The  measurable  effects  of  using  systems 
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engineering  in  planning  and  controlling  various  government 
R&D  projects  were  covered  by  Gerloff  (1973) .  Hovanessian 
(1975)  stated  the  desirability  of  applying  systems- 
engineering  methods  to  the  development  of  large-scale, 
military  electronics  systems.  These  several  references 
assisted  in  guiding  the  development  of  a  systems-engineering 
approach  to  be  applied  to  the  major-decision  milestones. 

This  chapter  sets  forth  the  existing  decision-process 
procedures,  identifies  the  value-system  operation,  and 
describes  the  nature  and  structure  of  the  process.  This 
chapter  also  identifies  the  criteria  and  organisational 
considerations  for  applying  the  systems-engineering  tools 
and  techniques  within  a  methodology.  The  areas  of  a  systems- 
acquisition  decision  process  where  the  application  of 
systems-engineering  might  pay  the  greatest  return  are 
identified.  An  appropriate  systems-engineering  approach 
is  developed  by  selecting  the  analysis  procedures  and 
associating  them  with  the  decision-process  support  areas. 

Finally,  the  developed  systems-engineering  approach  is 
applied  to  an  existing  DOD  systems  acquisition. 

The  material  in  this  chapter  was  developed  from  several 
sources.  The  referenced  technical  papers  and  Defense  documents 
established  a  foundation.  This  foundation  was  built  upon  with 
the  information  gained  through  personal  discussions  with  the 
people  listed  in  Appendix  B,  who  are  associated  with  Defense 
systems  acquisition.  The  information  gained  from  these  discussions 
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was  supplemented  by  data  from  a  number  of  working  papers  pro¬ 
vided  by  the  same  people.  Finally,  the  author's  four-years 
experience  in  Defense  R&D  provided  a  background  for  the  research. 

The  sources  noted  above  were  used  to  develop  a  mosaic 
of  the  Defense  systems-acquisition  process.  The  material  pre¬ 
sented  in  Existing  Procedures  and  Value  System  (p.  127)  and  The  Nature 
and  Structure  of  the  Defense  Systems-Acquisition  Process  (p.  144) is  a 
descriptive  senario  of  how  the  process  actually  works.  However, 
despite  the  validation  of  the  material  through  many  discussions 
with  DOD  systems-acquisition  personnel,  the  final  results  are 
from  the  author's  perspective.  A  description  of  the  process 
could  not  be  universally  accepted  by  all  parties  associated 
with  Defense  systems  acquisition. 

Although  the  descriptive  scenario  is  important,  the 
major  goal  is  to  develop  a  methodology  for  Defense  systems 
acquisition.  A  normative  scenario  of  how  the  process  should 
operate  is  presented  in  Development  of  Systems -Engineering  Methodology 
(p.  183) and  Implementation  of  the  Methodology  (p.  237)  . 

Existing  Procedures  and  Value  System 

The  two  DOD  directives,  5000.1  and  5000.2,  established 
the  basic  DOD  policies  for  systems  acquisition.  However,  a 
number  of  other  directives  also  set  forth  DOD  policies 
directly  related  to  the  systems-acquisition  process.  Included 
in  these  other  directives  were: 

1.  Defense  Acquisition  Executive ,  DOD  Directive  5000.30 
(Defense,  1976) ,  outlined  the  responsibilities  of  the  DAE; 
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2.  Planning ,  Programming  and  Budgeting  Systems , 

DOD  Directive  7045.7  (Defense,  1969),  set  forth  the  policies 
for  planning,  programming,  and  budgeting  system  (PPBS) ; 

3.  Test  and  Evaluation ,  DOD  Directive  5000.3  (Defense, 
1973d),  which  established  the  policies  for  T&E  in  the  acquisition 
of  Defense  systems; 

4.  Design  to  Cost ,  DOD  Directive  5000.28  (Defense, 
1976c)  ,  established  the  policies  for  the  application  of 
design-to-cost  principles; 

5.  Cost  Analysis  Improvement  Group ,  DOD  Directive 
5000.4  (Defense,  1976a),  which  outlined  the  policies  of  the 
cost-analysis  improvement  group  (CAIG)  in  relation  to  indepen¬ 
dent  cost  estimates  for  systems  under  development. 

Each  service  headquarters  implemented  the  policies 
of  DOD  Directives  5000.1  and  5000.2  by  issuing  its  own 
policy  directive.  The  variety  of  approaches  the  several 
services  have  used  to  implement  the  DOD  directives  is  shown 
in  the  policy  documents  that  each  has  issued.  For  instance, 
the  Air  Force  has  recently  issued  Statement  of  Operational 
Seed,  AFR  57-1  (USAF,  1978),  which  outlined  in  great  detail  the 
Air  Force's  policies  for  developing  and  processing  the  MENS; 
other  services  have  not  addressed  this  issue  in  such  detail. 

The  individual  services'  systems-acquisition  policies 
and  related  documentation  have  also  varied  in  the  procedures 
associated  with  the  DSAP.C/ ( S)  SARC .  The  Army  issued  Systems 
Acquisition  Review  Council  Procedures,  AR  15-14  (Army,  1978b), 
which  provided  detailed  guidance  governing  the  operation  of 


the  Army  Systems  Acquisition  Review  Council  (ASARC)  and  the 
Army's  involvement  in  the  DSARC.  These  policies  were  further 
amplified  by  an  Army  directive  from  the  Office  of  the  Deputy 
Chief  of  Staff  for  Research,  Development,  and  Acquistion 
(ODCSRDA)  ,  AS  ARC /DSARC  Procedures,  Regulation  15-14  (Army,  1978a)  , 
which  provided  detailed  procedures  within  the  ODCSRDA  for  the 
ASARC/DSARC  process.  The  other  services  have  not  addressed 
this  subject  in  such  detail. 

Justifications  for  these  various  approaches  are  related 
to  how  the  services  viewed  their  responsibilities  and  their 
relationships  with  OSD  and  subordinate  commands.  The  arguments 
for  these  approaches  are  not  of  interest  here,  but  differences 
do  exist,  not  only  in  policies  but  also  in  procedures. 

The  nature  of  the  programs  subject  to  review  through 
the  DSARC/ ( S ) S ARC  process  also  differ.  First,  there  are 
the  functional  differences  of  the  mission-area  needs,  such 
as,  the  development  of  the  FFG-7  Guided  Missile  Frigate 
required  for  surface-warfare  missions,  the  F-16  Fighter 
required  for  tactical-air  missions,  or  a  new  main  battle 
tank  required  for  close-combat  (ground  forces)  missions. 

Second,  the  program  could  be  divided  into  strategic  and  tacti¬ 
cal  programs.  Strategic  programs  could  cause  management 
problems  because  the  data  associated  with  the  programs  were 
limited  by  security  restrictions.  Third,  the  programs  could 
be  divided  into  new  development  programs,  as  was  the 
Missile  X  (MX)  program,  or  modification  programs,  as  was  the 
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Air  Force’s  cargo-plane  modification  C-141  (the  Stretch  program). 
Other  classification  systems  could  also  be  devised. 

The  differences  noted  above  are  significant  to  the 
services'  management  approaches  and  to  the  programs  themselves. 
However,  the  management  approaches  of  the  services  and  the 
way  the  various  programs  are  addressed  are  fairly  consistent 
because  of  the  parent  DOD  directives,  5000.1  and  5000.2. 

In  particular,  the  MENS  with  its  approval  cycle  and 
the  DCP  with  the  (S) SARC/SERVSEC  and  DSARC/SECDEF  approval 
cycle  unify  the  overall  Defense  systems-acquisition  process. 
These  requirements  have  encouraged  consistency  in  the  overall 
process  and  provided  substantative  activities  about  which  the 
systems-engineering  approach  can  be  developed. 

The  overall  procedure  for  developing,  processing, 
and  approving  the  MENS  is  graphically  portrayed  in  the  DELTA 
chart  of  Figure  16.  The  procedure  begins  with  a  continuing 
mission-area  analysis  by  the  service.  Based  on  this  analysis, 
a  new  mission-element  need  is  identified.  This  need  is 
documented  in  a  draft  MENS  by  the  requirements  command  of 
the  service.  The  draft  MENS  is  then  reviewed  by  the  service 
headquarters  and  either  approved  or  disapproved  by  the  Service 
Secretary.  The  draft  MENS  is  revised  as  deemed  appropriate  by 
the  several  review  authorities  and  then  forwarded  to  OSD  for 
review  by  the  OSD  staff  and  the  DAE.  After  this  review,  the 
draft  MENS  is  either  approved  or  disapproved  by  the  SECDEF . 

If  the  draft  MENS  is  approved ,  appropriate  actions  are  taken 


to  update  the  PPBS .  This  update  will  insure  that  the  PPBS 
(including  the  Program  Objectives  Memorandum  [POM]  and 
Five  Year  Defense  Plan  [FYDP] )  is  in  consonance  with  the 
systems-acquisition  plans.  Finally,  the  prog ram -management 
charter  is  issued  establishing  the  program-management  office. 

The  overall  cycle  for  the  development,  processing, 
and  approval  of  the  DCP  is  graphically  portrayed  in  the  DELTA 
chart  of  Figure  17.  The  cycle  begins  with  the  SECDEF '  s 
decision  that  ends  the  previous  phase  and  begins  the  new  phase. 
This  decision  is  documented  in  the  DCP  and  results  in  an 
update  to  the  PPBS  and  the  program-management  directive  for 
the  systems-program  office.  The  systems-program  office  then 
carries  out  the  next  phase  of  the  systems-acquisition  process 
culminating  in  the  recommendations  for  the  next  phase.  These 
recommendations  are  provided  to  the  service  headquarters, 
which  develops  a  draft  DCP.  The  chart  also  indicates  that 
independent  program  assessments  may  be  carried  out  external 
to  the  headquarters . 

The  draft  DCP  and  the  critical  issues  associated  with 
the  program  are  next  reviewed  by  (S)SARC  and  recommendations 
are  made  to  the  SERVSEC .  Appropriate  action  is  then  taken 
on  the  DCP  by  the  SERVSEC.  If  the  SERVSEC  approves  the  draft 
DCP,  it  is  then  forwarded  to  the  DAE  and  OSD  staff  for  review. 
The  CAIG  carries  out  independent  cost  estimates;  other  OSD 
staffs  can  also  conduct  independent  program  evaluations. 

The  DSARC  reviews  the  DCP  and  the  critical  issues  associated 


Figure  17.  Decision  Coordinating  Paper  (DCP)  Approval  Cycle,  including 
Independent  Evaluations. 
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with  the  program  and  makes  recommendations  to  the  SECDEF . 
Finally,  the  SECDEF  takes  action  on  the  DCP  and  issues  his 
decision  thus  completing  the  cycle. 

The  membership  of  the  DSARC  and  the  (S)SARC  is  impor¬ 
tant  for  an  understanding  of  the  overall  DOD  systems- 
acquisition  decision  process.  Comments  on  the  membership 
of  the  several  councils  are  presented  below. 

The  members  of  the  DSARC  are  senior  civilian  managers 
from  OSD,  including  the  DAE  (Chairman) /Under  Secretary  of 
Defense  for  Research  and  Engineering,  Assistant  Secretary  of 
Defense  (Comptroller) ,  Assistant  Secretary  of  Defense  (Manpower, 
Reserve  Affairs,  and  Logistics) ,  Assistant  Secretary  of  Defense 
(International  Security  Affairs) ,  Assistant  Secretary  of 
Defense  (Program  Analysis  and  Evaluation) ,  and  the  Special 
Advisor  for  NATO  Affairs.  Usually,  these  individuals  have 
had  previous  executive  experience  in  government,  industry, 
or  commerce  and  hold  a  graduate  degree  (or  its  equivalent) 
in  a  discipline  associated  with  the  particular  position.  For 
instance,  the  Under  Secretary  of  Defense  for  Research  and 
Engineering  has  traditionally  held  a  doctorate  of  science. 

The  membership  of  the  individual  (S)SARC  is  somewhat 
more  heterogeneous  than  the  membership  of  the  DSARC .  The 
Marine  Corps  Council  is  almost  exclusively  general  officers 
(Navy,  1979),  the  Army's  council  is  predominately 
general  officers  but  also  has  several  senior,  civilian  service 
secretaries  (Army,  1978a) ,  and  the  Air  Force  (USAF,  1977)  and 
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the  Department  of  the  Navy  (Navy,  1973)  have  about 

equal  representation  of  generals  or  flag  officers  and  senior, 

civilian  service  secretaries  as  members  on  their  councils. 

Many  of  these  individuals  hold  graduate  degrees  but  not 
necessarily  in  a  discipline  closely  associated  with  their 
current  position.  The  majority  of  these  officers  have 
previously  attended  a  top-level  military  school;  such  as, 
the  National  War  College,  Industrial  College  of  the  Armed 
Forces,  the  Army  War  College,  or  the  Naval  War  College. 
Normally  the  service  secretaries  have  previous  executive 
experience  and  the  officers  have  spent  many  years  in  both 
operational  and  staff  positions. 

In  studying  the  decision  process  of  Defense  systems 
acquisition,  the  value  system  of  the  DOD  (and  specifically 
the  DSARC)  must  be  considered — what  are  the  goals  of  systems 
acquisition  that  the  DOD  seeks  to  obtain.  The  motives  and 
intentions  of  the  DSARC  members  and  consequently  the  systems- 
acquisitions  questions  they  must  answer  are  influenced  by 
these  goals  and  by  the  background,  experience,  education,  and 
position  of  each  member. 

The  attainment  of  these  goals  is  limited  by  the 
following  considerations:  the  available  resources  (qualified 
DOD  personnel,  funds,  qualified  contractors,  facilities,  etc.), 
the  management  time  available,  the  political  environment  in 
the  federal  government,  and  the  existing  systems-acquisition 
policies.  The  alterables  to  be  considered  for  this  problem 
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are  the  management  information  provided  to  the  DSARC ,  the 
analysis  approaches  used  to  address  the  problems,  the 
OSD-established  procedures  for  preparing  and  executing  the 
DSARC,  and  the  policies  associated  with  the  overall  process. 
Similar  sets  of  constraints  and  alterables  exist  for  the 
(S)SARCs.  The  means  by  which  the  achievement  of  a  goal  can 
be  measured  is  discussed  in  Implementation  of  the  Methodology 
(p- 237)  . 

At  the  top-most  level  of  objectives,  the  DOD  may 
simply  be  considered  an  instrument  to  pursue  national  policies 
The  acquisition  of  Defense  systems  can  be  viewed  as  supporting 
this  objective  since  the  systems-acquisition  process  provides 
the  capabilities  to  implement  the  policies.  President  Carter 
addressed  such  a  top-level  goal  at  the  NATO  summit  meeting 
in  London  in  May  of  1977,  when  he  stated, 

We  must  combine,  coordinate,  and  concert  our  national 
programs  more  effectively.  We  must  find  better  ways  to  bring 
new  technology  into  our  armed  forces.  We  must  give  higher 
priority  to  increasing  the  readiness  of  these  forces. 

(Freeman,  1978) 

DSARC  members  review  the  costs  and  benefits  of  various 
acquisition  programs  while  attempting  to  achieve  several 
objectives.  They  must  be  responsive  to  the  lead  of  Congress 
and  to  the  guidance  of  the  DOD  in  systems-acquisition  matters. 
They  need  to  be  concerned  with  the  maintenance  of  a  viable 
Defense  industry  in  the  United  States  (Gansler,  1977)  and 
with  a  strong  systems-acquisition  management  capability  in 
the  services  (Packard,  1971).  International  security  matters 


must  also  be  considered.  An  example  is  the  expressed  concern 
of  Dr.  William  J.  Perry,  then  Under  Secretary  of  Defense  for 
Research  and  Engineering,  for  the  Russian  capability  to 
destroy  incoming  cruise  missiles  (Klass,  1978). 

In  attempting  to  develop  the  best  overall  program 
of  systems  acquisitions,  the  DSARC  members  are  affected  by 
their  OSD  staff  positions  and  their  own  personal-preference 
goals.  The  objectives  being  sought  by  the  members  include  the 
achievement  of  NATO  interoperability  and  standardization 
(Freeman,  1978) ,  the  documentation  of  the  MENS  for  various 
acquisition  programs  (Defense,  1978b)  and  the  identification 
of  critical  issues  and  evaluation  criteria  for  each  program. 

Several  recommendations  were  presented  in  the  Defense 
Science  Board  Task  Force  Report  (Defense,  1978c)  on  systems 
acquisitions.  The  objectives  based  on  this  report  are:  to 
minimize  the  length  of  the  development  cycle  for  acquisition 
and  to  improve  the  initiation  of  the  process,  to  establish 
a  minimum  set  of  programs  necessary  to  meet  national 
defense  requirements,  and  to  make  systems-acquisition  processes 
more  flexible. 

The  objectives  appropriate  for  consideration  by  the 
DSARC  members  have  been  formed  into  an  objectives  tree 
(Figure  18) .  The  objectives  at  each  level  of  this  tree 
contribute  to  the  accomplishment  of  the  objective  on  the  next 
higher  level.  The  objectives  become  more  specific  and  concrete 
as  one  moves  down  the  tree. 


Figure  18.  Objectives  Tree  for  the  Value  System  Operating  in  the  DSARC. 


A  similar  objectives  tree  could  be  developed  for  the 
members  of  the  (S)SARCs.  The  objectives  tree  for  the  DSARC 
and  the  one  for  the  (S)SARCs  would  be  quite  similar;  most 
changes  would  simply  reflect  the  services'  subordination  to 
the  SECDEF .  However,  there  would  be  one  significant  dif¬ 
ference  in  the  (S)SARC  objectives  tree  because  the  members 
of  the  ( S ) S ARC  are  motivated  by  the  institutional  values 
of  their  respective  services.  They  are  concerned  with  the 
survival  of  their  service  and  the  perpetuation  of  the 
traditional  roles  and  missions  of  their  service,  which  is 
reflected  in  their  wish  to  obtain  systems  that  support 
these  missions.  (For  instance,  the  pursuit  of  the  develop¬ 
ment  program  for  main  battle  tanks  by  the  Army  and  tactical 
aricraft  by  the  Air  Force.)  An  objective  on  the  3. — level 
and  its  subordinate  objectives  should  incorporate  this 
institutional  advocacy  into  the  (S)SARC  objectives  tree. 

The  objectives  for  the  DSARC  and  (S)SARCs  and  the 
DOD  activities  associated  with  them  are  functionally 
related  in  such  a  way  that  inclusion  in  a  simple  objectives- 
tree  format  might  be  considered  inappropriate.  For  instance, 
several  feedback  and  feed-forward  loops  might  be  identified 
between  tne  objectives  presented  in  Figure  18.  However, 
the  purpose  here  is  to  present  the  goals  of  the  DSARC  and 
(S)SARC  members  in  the  most  straight-forward  manner  possible; 
therefore  the  objectives  tree  was  chosen  as  the  best 
approach  to  display  the  goals . 
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The  procedures  by  which  the  services  prepare  for 
their  own  (S)SARC  and  the  DSARC  are  also  of  interest  in 
this  study.  In  reviewing  the  relevant  directives  and  in 
discussions  with  the  personnel  involved,  a  great  deal  of 
variance  was  observed  in  the  approaches  taken  by  the  ser¬ 
vices  to  these  preparations.  The  Army's  approach  is  well 
documented,  highly  structured,  and  apparently  closely 
adhered  to.  The  other  services  are  somewhat  less  formal 
in  their  approach. 

A  detailed  procedural  model  for  the  preparations 
for  and  execution  of  the  DSARC/ (S) SARC (s)  that  would  be 
applicable  to  all  services  cannot  be  developed.  However, 
a  generalized  set  of  procedures,  not  necessarily  in  se¬ 
quence,  for  these  preparations  are  presented  as  follows: 

1.  Program  alternatives  with  the  supporting 
documents  are  developed  and  critical  issues  are  identified 
within  the  program  office. 

2.  Personnel  at  commands  subordinate  to  the  service 
headquarters  are  briefed  on  the  alternatives. 

3.  Preparations  for  the  (S)SARC  are  coordinated 
at  the  service  headquarters. 

4.  Additional  program  information  is  obtained  from 
other  commands  within  the  serivce  (i.e.,  T&E  data  from  the 
test  command. 

5.  A  variety  of  independent  analyses  are  carried 
out  by  service  headquarters. 
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6.  Prebriefings  are  held  for  the  principals  of 
the  (S)SARC. 

7.  The  DCP  is  developed  or  updated  at  the  service 
headquarters  and  the  (S)SARC  is  scheduled. 

8.  The  (S) SARC  is  held;  the  program  manager  makes 
a  formal  presentation  of  the  mission-element  need,  program 
status,  critical  issues,  and  projected  alternatives  for 
the  next  phase;  and  a  specific  alternative  is  recommended. 

9.  The  (S)SARC  members  make  a  recommendation  that 
is  forwarded,  with  the  draft  DCP,  to  the  Service  Secretary 
for  a  decision. 

10.  The  SERVSEC  decides  on  the  critical  issues; 
if  the  program  is  approved  for  continuance,  and  the  draft 
DCP  is  then  forwarded  to  OSD  with  the  SERVEC's  recommen¬ 
dation  . 

11.  A  CAIG  independent  cost  estimate  and  any  other 
required  independent  evaluations  are  submitted. 

12.  The  SERVSEC' s  decision,  the  draft  DCP,  and  the 
supporting  documents  are  reviewed  by  the  DAE  and  OSD  staff. 

13.  A  pre-DSARC  meeting  with  the  DSARC  principals 
is  held  on  the  specific  program. 

14.  The  DSARC  is  then  convened  and  the  program  manager 
a  formal  presentation  of  the  mission-element  need(s), 

-  •  i-  status,  critical  issues,  and  projected  alternatives 

-  - -  -  xt  phase.  A  specific  recommendation  is  made  as 
• *rnative  should  be  chosen. 
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15.  The  DSARC  members  make  a  recommendation  that 

is  forwarded,  with  the  draft  DCP,  to  the  SECDEF  for  a  decision. 

16.  The  SECDEF  decides  on  the  critical  issues;  if 
the  program  is  approved  for  continuance,  the  final  version 
of  the  DCP  is  returned  to  the  service  headquarters  for 
implementation  of  the  next  phase. 

Although  the  program  manager  usually  makes  the  formal 
presentation  at  the  reviews,  others  may  assist.  (For  example, 
the  Army  requires  that  representatives  from  its  Training 
and  Doctrine  Command  [TRADOC] ,  The  Army's  user,  present 
the  briefing  of  the  MENS,  threat  assessment,  and  opera¬ 
tional  concept.)  The  time  allowed  for  the  overall 
presentation  is  usually  limited  to  prevent  the  briefings 
from  being  too  lengthy  and  the  technical  content  is1  kept 
to  a  minimum. 

A  set  of  viable  alternatives  are  usually  presented 
to  the  DSARC  or  (S)SARC.  The  program  manager  should  present 
the  key  factors  of  each  alternative  and  recommend  a  specific 
one.  Although  there  may  be  an  obvious  preferred  alternative, 
the  other  alternatives  should  be  feasible  and  capable  of 
being  implemented;  presenting  a  very  strong  recommended 
solution  and  very  weak  alternatives  is  inappropriate. 

The  DSARC/ (S) SARC  members  may  choose  an  alternative 
in  a  variety  of  ways.  The  members  may  actually  vote  on  a  set 
of  alternatives  or  they  may  be  polled  to  determine  if  they  are 


dissatisfied  with  the  recommended  alternative.  The  members 
may  present  dissenting  or  minor  opinions  as  part  of  the 
decision  forwarded  to  the  SERVSEC  or  the  SECDEF .  Although 
unusual,  because  of  the  political  environment  within  DOD ,  the 
Chairman  could  simply  direct  a  decision.  The  rules  used  to 
arrive  at  a  particular  decision  (recommendations  to  the 
SERVSEC  or  SECDEF)  differ  with  each  council  and  each  program. 
However,  this  decision  process  is  a  group-decision  process  in 
the  context  of  decision  analysis  (see  Raiffa,  1968).  The 
topic  of  decision  analysis  is  addressed  again  later  in  this 
chapter . 

Two  final  areas  are  important  to  the  overall  decision 
process — the  changing  nature  of  the  alternatives  and  the 
evolution  of  a  decision  process  within  the  service  headquarters 
and  OSD.  When  a  particular  alternative  is  first  identified, 
it  does  not  have  any  particular  status.  But,  as  it  becomes 
more  clearly  defined  and  identifiable,  it  tends  to  gain  a  life 
of  its  own.  Advocates  come  forward  from  a  variety  of  areas, 
industry,  users,  and  developers,  to  support  the  alternative. 
After  analysis  and  evaluation,  a  single  alternative  becomes 
the  recommended  alternative  at  the  program-office  level. 

The  alternative  may  then  be  supported  at  one  or  more  higher 
levels.  (For  instance.  Air  Force  Systems  Command  normally 
reviews  a  program  before  it  briefs  the  Air  Force  Headquarters.) 
The  recommended  alternative  has  picked  up  support  from  several 
more  echelons  of  command. 


Ultimately  the  (S)SARC  is  briefed  on  the  alternatives, 
a  single  alternative  is  recommended  to  the  SEP.VSEC,  and  a 
recommended  alternative  is  forwarded  to  OSD  for  review  and 
consideration  by  the  DSARC .  At  this  stage,  the  recommended 
alternative  is  no  longer  simply  a  program-office  recommendation; 
rather,  it  has  often  taken  on  the  status  of  a  service  posi¬ 
tion — the  result  of  a  service  assuming  the  role  of  sponsor 
and  advocate  for  the  recommended  alternative.  An  approach  of 
this  type  could,  unless  carefully  controlled,  minimize  other 
viable  alternatives. 

The  final  comments  here  concern  the  evolution  of  the 
decision  process  for  systems  acquisitions  within  the  DOD. 

First,  the  SERVSECs  and  the  SECDEF  can  and  do  revise  the 
recommendations  of  the  (S)SARCs  and  DSARC.  Second,  several 
decisions  have  been  well  on  the  way  to  being  completed  prior 
to  the  ( S )  SA.RC  or  DSARC  proceedings  because  of  activities 
outside  the  DOD — Congressional  action,  executive-department 
decisions,  or  some  other  reason.  Third,  DOD  systems- 
acquisition  decisions  are  not  confined  to  the  DSARC/ (S) SARCs . 
Although  the  DOD  policy  directives  for  systems  acquisition 
imply  that  the  DSARC/ (S) SARC  process  works  almost  unilaterally 
on  systems  acquisition,  this  is  not  the  case.  A  number  of 
working  groups,  committees,  and  councils  within  the  several 
services  have  a  great  affect  on  the  systems-acquisition 
process,  particularly  in  funding;  such  as,  the  development 
of  the  POM  and  FYDP ,  as  each  service  carries  out  its  PPBS 
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responsibilities.  The  Chief  of  Staff's  Committees  within 
the  Marine  Corps  and  the  Air  Council  within  the  Air  Force  are 
examples.  These  top-level  review  groups  are  staffed  by  senior 
general  officers  and  significantly  influence  many  decisions, 
including  those  involving  funding  and  operations. 

The  DSARC/ (S) SARC  process  is  not  the  single  path 
through  which  systems-acquisition  decisions  are  made.  In 
some  cases  the  DSARC/ ( S) SARC  process  becomes  a  validation 
process  for  decisions  that  basically  have  already  been  made. 
However,  the  policies  of  the  DOD  and  the  several  services 
do  point  toward  the  DSARC/ (S) SARC  process  as  the  principal 
means  by  which  systems-acquisition  programs  are  reviewed 
and  the  recommended  decisions  are  developed  for  the  SERVSECs 
and  SECDEF .  Also,  personnel  involved  with  the  systems- 
acquisition  process  within  the  DOD  want  to  make  the  DSARC/ 
(S)SARC  process  as  viable  and  meaningful  as  possible  for 
the  various  stakeholders  of  the  process,  including  the  services, 
the  executive  branch,  Congress,  the  Defense  industries,  and 
the  American  people. 

The  Nature  and  Structure  of  the  Defense 
Sy stems -Acquisition  Process 

The  projected  benefits  from  implementing  OMB  circular 
A-109 ,  Major  Systems  Acquisition,  and  the  two  revised  DOD 
directives  Major  Systems  Acquisition,  5000.1,  and  Major 
Systems  Acquisition  Process,  5000 . 2  ,  are  presented  in  an  article 
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by  Mr.  P.  R.  Calaway,  Assistant  for  Program  Planning  in  the 
Office  of  the  Under  Secretary  of  Defense  for  Research  and 
Engineering  (Calaway,  1978).  Calaway  stated,  in  part, 

A-109  is  here  to  stay.  We  agree  with  its  principles, 
and  we  support  it.  Why?  Because  we  think  it  makes  sense, 
and  because  we  think  it  has  the  potential  to  help  us  shorten 
the  acquisition  cycle  and  get  us  more  capability  for  our 
dollar. 

These  comments  gave  a  rather  positive  view  of  the 
new  systems-acquisition  policies.  However,  the  implementa¬ 
tion  of  A-109  has  not  been  carried  out  without  problems.  The 
Under  Secretary  of  Defense  for  Research  and  Engineering 
covered  some  of  these  problems  in  a  memorandum  to  senior 
DOD  personnel  (Defense,  1978b).  The  memorandum  stated 
that  the  DOD  had  failed  to  offer  Congress  the  opportunity 
to  debate  key  mission  needs  before  resources  were  committed 
to  the  identification  of  solutions,  that  there  was  concern 
about  the  intent  of  the  MENS  in  the  systems-acquisition 
process,  that  what  constitutes  a  MENS  was  unclear,  and  that 
which  of  the  programs  required  a  MENS  was  confusing.  Secre¬ 
tary  Perry  addressed  each  issue  and  set  forth  additional 
policies  in  the  memorandum  to  clarify  the  requirements  of 
the  OMB  and  5000.1  directives. 

Secretary  Perry  did  not  address  the  root  causes  of 
problems  in  his  memorandum.  However,  these  problems  were 
probably  caused  by  an  institutional  inertia  to  address  systems- 
acquisition  problems  in  the  traditional  way,  the  failure  to 
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penetrate  the  DOD  bureacracy  with  new  procedures,  and  an  honest 
misunderstanding  as  to  what  was  desired  from  the  services. 

It  is  appropriate  to  note  the  magnitude  of  the  problem 
of  developing  MENS  for  each  of  the  major  systems-acquisition 
programs.  First  there  are  many  indications  that  the  services 
view  the  development  of  each  MENS  as  a  major  effort  because 
of  a  lack  of  experience  and  a  lack  of  understanding  of  what 
OSD  needs  and  wants.  Second,  although  fifty-seven  major 
systems-acquisition  programs  required  DSARC  consideration 
as  of  November,  1978  (Defense,  1979),  only  five  MENS  had 
been  approved  at  the  OSD  level  by  April,  1979.  This  observa¬ 
tion  of  MENS ( s)  was  made  almost  three  years  after  the  OMB 
circular  was  issued  and  over  two  years  after  the  original 
DOD  directives  we re  revised. 

During  1977-78,  the  Defense  Science  Board  Task  Force 
studied  the  implementation  of  the  systems-acquisition  policies 
in  the  OMB  circular  (Defense,  1978c).  This  task  force 
included  members  from  industry,  the  OSD,  and  the  military 
services.  Their  report  was  written  at  the  request  of  the 
Under  Secretary  of  Defense  Research  and  Engineering  and 
included  the  following  findings  and  conclusions: 

1.  No  Sense  of  Uvgenoy. 

It  takes  longer  to  get  things  done  in  the  DOD  (and  else¬ 
where  in  the  U.S.)  than  it  used  to.  The  increased  delays 
seem  to  be  in  the  decision  process  rather  than  in  the  time 
to  do  the  actual  work. 
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2.  Flexibility. 

The  most  prominent  single  thread  that  was  evident  through¬ 
out  all  of  the  data  examined  by  the  Task  Force  was  the  necessity 
for  and  absence  of  a  high  degree  of  flexibility  in  every  appli¬ 
cation  of  the  policies  and  practices  for  acquisition  management. 

3.  Motivation.  The  task  force  perceived  that 

both  institutional  and  personal  values  have  a  substantial 
influence  on  the  acquisition  process. 

4.  Program  Advooaoy .  The  Task  Force  noted  that  there 
was  a  high  correlation  between  the  programs  that  had  strong 
advocacy  and  the  programs  that  were  continued  in  the  develop¬ 
ment  cycle.  The  programs  that  survived  had  active  and  vocal 
advocates  in  the  sponsoring  service,  in  OSD,  in  Congress,  or 
elsewhere . 

5.  Concurrency.  The  task  force  defined  concurrency  as 

the  conduct  of  the  steps  leading  to  production  for  inventory 
before  the  end  of  the  full-scale  development  time  span. 

The  task  force  found  that  a  certain  amount  of  concurrency  in 

a  program  could  shorten  the  acquisition  process  and  provide  a 

savings  over  the  total  acquisition  process.  This  finding  was 

based  on  the  task  forces'  conclusion  that 

a  commitment  to  enter  full-scale  development  should  be 
recognized  and  accepted  as  a  clear  reaffirmation  of  intent 
to  produce  and  deploy  the  developed  article,  barring  truly 
unforeseen  events. 

The  task  force  further  concluded  that  the  decision  for  full- 
scale  development  (Milestone  II)  was  second  in  importance 
only  to  the  Milestone  0  decision,  approval  of  MENS  (with 
intent  to  produce  and  deploy) . 
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6.  Prototypes.  The  task  force  also  investigated  the 
use  of  preproduction  prototypes  before  stating  that 

prototyping  can  be  a  sound  and  useful  practice  in  major 
systems  acquisition  provided  that  the  candidates  for  the  use  of 
prototypes  are  carefully  selected,  that  only  those  things 
are  prototyped  which  really  need  verification,  and  that  pro¬ 
totypes  are  not  considered  to  be  some  form  of  free  lunch,  for 
the  procuring  agency. 

7.  Operational  Test  and  Evaulation .  The  task  force 
identified  a  flexible  testing  philosophy  as'  a  prerequisite  to 
improving  the  acquisition  process.  The  desirability  of 
joint  testing  but  independent  evaluation  and  the  need  for 
close  cooperation  between  the  users  and  the  developers 

was  also  discussed. 

8.  Early  Deployment .  The  task  force  concluded  that 

there  is  considerable  evidence  to  support  the  claim  that 
early  deployment  is  frequently  a  useful  and  valuable  practice, 
particularly  in  those  cases  where  less  than  the  ultimate  system 
performance  is  acceptable  in  the  initially-deployed  units. 

The  task  force  also  concluded  that  early  deployment  could 

save  significant  program  costs  in  terms  of  reduced  inflation, 

fewer  engineering  change  proposals  for  gold-plating ,  and  a 

minimization  of  overhead  costs. 

9.  The  External  Environment.  The  task  force  further 
concluded  that 

due  to  the  influence  of  external  forces,  the  longer  a 
program  stays  in  full-scale  development,  the  greater  are  its 
chances  of  being  cancelled  prior  to  completion. 

This  was  caused  by  personnel  and  viewpoint  changes,  DOD,  the 

Congress,  the  Executive  Branch,  and  the  public  sector.  These 

changes  result  in  a  shift  of  the  priorities,  attitudes,  and 
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appreciation  of  the  enemy  threat  that  caused  the  program  to 
be  approved  for  development  in  the  first  place.  The  recent 
cancellation  of  the  B-l  bomber  program  is  an  example  of 
this  phenomenon. 

10.  Increasing  Realism  in  Cost  Estimation.  The  task 
force  concluded  that  many  considerations  were  important  in 
planning  program  costs,  including  allowances  for  additional 
tasks,  a  need  for  all  concerned  to  help  get  the  job  done,  the 
selection  of  sources  based  on  the  optimism  of  contractors 
rather  than  realism,  and  the  impact  of  inflation  on  lengthy 
programs . 

Based  on  the  above,  the  task  force  recommended  several 
acquisition-policy  initiatives.  These  initiatives  are 
reflected  in  the  DSARC/ (S) SARC  objectives  trees  (see 
Figure  18)  and  are  presented  below: 

1.  Reduce  the  Number  of  Programs.  It  was  recommended, 

That  the  number  of  major  weapon-systems-development  pro- 
. grams  should  be  reduced  so  that  those  which  are  continued  are 
the  ones  which  DOD  intends  to — and  can  afford  to — put  into 
production  and  deployment. 

2.  Demand  and  Approve  a  Flexible  Approach.  The  task 
force  recommended  a  flexible  approach  to  the  systems-acquisition 
process  that  would  include  a  broad,  generalized  approach  at 
the  policy  level,  basing  competitive  selections  on  achieve¬ 
ment;  the  careful  development  of  the  MENS  to  enable  a  variety 
of  alternatives  to  respond  to  it;  the  use  of  the  DSARC  for 
decisions  and  program  reviews;  and  the  adjustment  of  thresholds 
for  programming  funds. 
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3.  Improve  and  Shorten  the  Front  End  of  the  Cycle. 

The  task  force  also  recommended  that 

the  formal  steps  in  the  definition  and  approval  of  an  acquisi¬ 
tion  program  be  reduced  wherever  appropriate  to  the  needs  of 
the  individual  program,  either  by  complete  elimination  of 
certain  steps  or  by  their  combination  with  other  milestones. 

The  comments  with  this  recommendation  indicated  that  recent 

policy  changes  have  elongated  rather  than  shortened  the  mission- 

analysis  (program  initiation  and  definition)  phase. 

The  Defense  Science  Board  Task  Force  '  s  recommendations 
were  reviewed  by  the  DEPSECDEF  after  which  he  recommended 
that  his  subordinates  in  the  DOD  study  the  report  for  possible 
implementation  of  the  recommendations  (Defense,  1978a).  The 
DEPSECDEF  directed  special  attention  to  the  recommendation 
"to  improve  the  acquisition  process  by  reducing  the  length 
and  cost  of  the  cycle  from  initial  conception  to  operational 
deployment. " 

Industry,  too,  has  been  concerned  about  the  implementa¬ 
tion  of  OMB  A-109 .  One  of  industry's  principal  spokesmen, 

Mr.  John  H.  Richardson,  President  of  Hughes  Aircraft  Company, 
has  demonstrated  a  continuing  interest  in  Defense  systems- 
acquisition  as  indicated  by  his  comments  in  Government 
Executive  (December,  1974;  February,  1976),  his  position  on 
the  Board  of  Visitors  of  the  Defense  Systems  Management  College, 
and  his  frequent  appearances  as  a  guest  speaker  at  the  college. 
In  April,  1978,  Richardson  testified  about  OMB  A-109  before 
the  Subcommittee  on  Research  and  Development,  Committee  on 
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Armed  Services,  U.  S.  House  of  Representatives,  on  behalf 
of  the  Aerospace  Industries  Association  of  America,  Inc. 
(Congress,  1978).  Initially  he  pointed  out  that  the  aero¬ 
space  manufacturers  strongly  supported  the  thrust  and  most 
of  the  particulars  of  OMB  A-109.  However,  he  also  pointed 
out  several  potential  problems  that  the  industry  perceived 
with  the  implementation  of  A-109 . 

First,  he  expressed  concern  that  there  would  not  be 
adequate  funding  of  the  competitive  Concept  Formulation  Phase 
following  Milestone  0  and  all  subsequent  phases.  If  this 
happened,  then  industry  would  have  to  rely  on  the  Independent 
Research  and  Development  (IR&D)  funds  provided  by  the  federal 
government  for  creative  R&D  within  the  companies.  Richardson 
suggested  that  this  could  lead  to  a  significant  and  dangerous 
degradation  of  the  technology  base. 

Second,  he  was  concerned  that  overspecification  of 
the  program  would  undercut  the  intent  of  the  OMB  circular. 

He  felt  that  there  were  too  many  inhibiting  details  and  speci¬ 
fications  in  the  MENS,  the  Request  for  Proposal  (RFP) ,  and 
the  contract.  These  documents  could  be  stumbling  blocks  to 
the  smooth  forward  progress  of  the  program. 

Third,  he  stated  that  OMB  A-109  could  lengthen  the 
acquisition  process  rather  than  shorten  it  because  of  the 
possible  systems-acquisition  problems  identified  by  the  Defense 
Science  Board  Task  Force  Report  (1978a) .  He  supported  the 
task  force's  recommendations  for  a  flexible  approach  to  these 
problems . 
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Richardson  reiterated  these  problems  in  his  briefing 
at  the  National  Conference  of  the  American  Insititute  of 
Aeronautics  and  Astronautics  in  Los  Angeles,  California 
on  November  16,  1978.  In  particular,  he  spoke  of  the  growth 
of  the  projected  development  time  for  Defense  programs  under 
various  acquisition  policies  over  the  last  twenty  years 
(Figure  19) .  Re  projected  a  possible  cycle  of  sixteen  years 
from  approval  of  the  MENS  to  an  initial  operational  capability 
(IOC) .  Richardson  argued  that  the  adoption  of  a  flexible 
system-acquisition  approach  as  recommended  by  the  Defense 
Science  Board  Task  Force  could  reduce  the  acquisition  cycle 
to  from  six  to  eight  years  by  the  1980's.  Figure  20 
contrasts  the  two  approaches. 

Thus  far  in  this  section  the  implementation  of  A-109 
has  been  viewed  from  several  prospectives — that  of  the  Under 
Secretary  of  Defense  Research  and  Engineering,  the  Members 
of  the  Defense  Science  Board  Task  Force,  and  a  senior 
representative  of  industry.  From  these  comments,  the  following 
observations  of  the  systems-acquisition  process  should  help 
to  develop  an  applicable  systems-engineering  methodology: 

1.  A  mission-element  need  is  identified  almost 
randomly;  as  a  result  the  draft  MENS  is  processed  and  evaluated 
in  a  serial  process  rather  than  in  a  parallel.  Developing 
the  best  overall  DOD  systems-acquisition  program  could  be 
better  achieved  if  the  draft  MENS(s)  were  batchprocessed  for 
evaluation. 
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Figure  19.  Growth  of  Projected  Development  Time  for  Defense  Programs 
under  Various  Acquisition  Policies  (Richardson,  1978). 
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Figure  20.  A-109  (OMB,  1976)  Institutionalized  Typical  Program,  contrasted 

with  a  More  Flexible  Approach  (Richardson,  1978) 


2.  It  is  very  difficult  to  trade  off  any  given 
program  against  other  programs  because  of  the  random  develop¬ 
ment  of  the  draft  MENS(s)  as  a  function  of  time. 

3.  A  MENS  has  not  been  developed  for  all  programs. 

4.  All  programs  should  be  subjected  to  (or  clearly 
exempted  from)  the  MENS  process  to  establish  a  uniformity  of 
policy  application  and  equity  in  evaluation  and  selection. 
Currently  there  is  a  feeling  that  those  programs  that  have 
undergone  the  MENS  process  have  suffered  some  ill-defined 
penalty  to  which  the  other  programs  have  not  been  subjected. 

5.  Since  the  MENS  presents  the  results  of  the 
problem-definition  phase,  it  is  very  critical  to  the  overall 
cycle. 

6.  Once  the  MENS  is  approved,  the  program  begins  a 
life  of  its  own  and  is  very  difficult  to  cancel. 

7.  Developing  a  MENS  could  add  an  additional  front- 
end  to  the  programs,  thereby  elongating  the  acquisition  proces 

8.  The  decision  options  become  fewer  and  fewer  as 
the  program  moves  through  the  acquisition  cycle. 

9.  It  is  desirable  and  beneficial  to  move  the 
programs  through  the  acquisition  process  (and  hence  the 
major-decision  milestones)  as  speedily  as  possible,  consistent 
with  good  judgment. 

10.  The  nature  of  the  decisions  required  at  each  major 
decision  milestone  is  different: 
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a.  Milestone  0  (Program  Initiation  and  Advancement 
to  Conceptual  Phase) .  At  this  point,  limited  quantative  data 
are  available  on  such  items  as  the  threat  assessment  and  the 
existing  capabilities.  However,  this  decision  is  a  go/no-go 
decision  to  initiate  a  program  based  primarily  on  its 
qualitative  aspects.  When  this  decision  is  addressed, 
nature,  structure,  and  final  outcome  of  the  program  are 
questionable.  The  key  questions  to  be  answered  are:  (1)  Is 

a  program  required  to  answer  the  mission  need?  (2)  Is  this 
program  of  greater  value  to  the  national-defense  effort  than 
other  available  programs  or  investments?  It  is  very  difficult 
to  trade  off  a  specific  program  against  other  possible  programs 
because  of  the  time  variation  of  the  identification  of  a 
mission-element  need. 

b.  Milestone  I  (Approval  for  Advancement  to  the 
Demonstration- and- Validation  Phase) .  More  quantative  informa¬ 
tion  will  be  available  on  the  program  at  this  point  but 
uncertainty  and  the  associated  risks  remain  quite  high. 

Critical  issues  are  addressed  and  alternative  candidate 
solutions  are  evaluated.  The  alternatives  are  traded  off 
among  themselves,  and  several  alternatives  are  selected  for 
continuation  into  the  next  phase.  This  evaluation-and- 
selection  process  begins  in  the  program  office  and  culminates 
with  a  decision  from  the  SECDEF. 

c.  Milestone  II  (Approval  for  Advancement  to  the 
Full-Scale  Development  Phase) .  The  quantative  information 
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begins  to  dominate  the  qualitative  aspects  of  t'.ie  program; 
uncertainty  and  its  associated  risk  are  reduced.  Critical 
issues  are  addressed  and  the  alternate  solutions  are  evaluated; 
the  remaining  alternate  candidates  are  traded  off  among  them¬ 
selves.  A  specific  alternative  will  be  selected  for  full- 
scale  development  in  much  the  same  way  as  the  Milestone  I 
decision  was  made.  The  evaluation-and-selection  process  will 
begin  in  the  program  office  and  culminate  with  the  decision  of 
the  SECDEF .  Because  a  program  should  not  be  permitted  to 
enter  full-scale  development  unless  it  will  probably  go  to 
production, this  decision  is  second  in  importance  only  to  the 
program-initiation  decision. 

d.  Milestone  III  (Approval  for  Production  and 
Deployment) .  At  this  point  reliable  quantitative  data  should 
be  available  on  almost  all  aspects  of  the  program.  Therefore, 
program  uncertainty  and  its  associated  risks  should  be  minimal. 
This  decision  is  only  slightly  less  critical  than  the  Milestone 
II  decision  since  it  represents  a  final  commitment  by  the  ser¬ 
vice  to  produce  and  deploy  the  system.  Critical  issues  are 
considered  and  the  decision  is  basically  a  go/no-go  decision 
to  produce.  No  alternative  candidate  solutions  are  evaluated 
or  alternatives  selected.  The  recommendation  to  go  or  not 
to  go  into  p  oduction  begins  in  the  program  office  and  cul¬ 
minates  with  a  decision  from  the  SECDEF. 

11.  There  are  several  DOD  systems-acquisition  policy 


initiatives  that  should  be  implemented: 
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a.  That  a  more  flexible  approach  be  adopted 
for  the  overall  process. 

b.  That  the  policies  outlined  in  OMB  Circular 
A-109  be  integrated  with  the  programming  requirements  of 
PPBS  and  the  Congressional  budget  cycle. 

c.  That  a  MENS,  the  critical-requirements 
document  for  each  program,  clearly  define  the  problems. 

d.  That  the  mission-analysis  effort  prior  to 
Milestone  0  be  clearly  identified  as  a  phase  of  the  systems- 
acquisition  process.  (This  is  a  service  responsibility  for 
requirements  determination  but  also  a  problem-definition 
phase  in  the  overall  DOD  process.) 

e.  That  a  procedure  be  developed  to  allow  for 
the  evaluation  and  trade  off  of  programs  early  in  the  acqui¬ 
sition  cycle  at  the  DOD  level.  (This  procedure  would  allow 
for  the  determination  of  the  best  overall  systems- 
acquisition  program  for  the  DOD.) 

The  overall  structure  cf  the  systems-acquisition 
problem  should  also  be  investigated.  First,  the  preceding 
portion  of  this  dissertation,  where  the  systems-engineering 
morphology  was  used  to  model  the  systems  acquisition  process, 
is  reviewed.  Then,  the  systems-acquisition  process  is 
examined  to  see  its  possible  relationship  to  the  Stage 
Approach  for  evaluating  and  selecting  R&D  projects  as 
outlined  by  Albala  (1976). 
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In  the  previous  chapter,  the  systems-acquisition 
process  was  compared  to  the  morphology  of  systems  engineering 
(see  Hall,  1969)  .  This  morphology  is  a  framework  consisting 
of  three  dimensions:  time,  logic,  and  knowledge.  The  time 
dimension  includes  the  coarse-grain  phases  that  characterize 
the  systems  work  from  the  initial  conception  of  a  problem 
through  the  retirement  of  a  system.  The  logic  dimension 
deals  with  the  steps  or  activities  that  are  carried  out  in 
each  systems-engineering  phase.  The  knowledge  dimension 
refers  to  the  specialized  knowledge  of  the  different  profes¬ 
sions  and  disciplines  required  for  systems  works. 

The  time-dimension  phases  from  the  systems-engineering 
morphology  encompass  program  planning,  project  planning, 
systems  development,  production,  distribution,  operation, 
and  retirement.  These  phases  are  compared  to  the  Defense 
systems-acquisition  phases  as  shown  in  Table  6 .  The  phases 
of  the  morphology  are  closely  related  to  the  phases  of  the 
systems-acquisition  process. 

The  logic-dimension  steps  of  the  systems-engineering 
morphology  consist  of  problem  definition,  value-system  design, 
system  synthesis,  systems  analysis  and  modelling,  optimization, 
decision  making,  and  planning  for  action.  These  steps  are 
compared  to  the  activities  of  the  several  phases  of  the 
systems-acquisition  process  as  shown  in  Table  7.  The  activi¬ 
ties  from  each  phase  correlate  favorably  with  the  steps  of 
the  logic  dimension.  Justification  for  this  observation  is 


TABLE  6 


THE  TIME-DIMENSION  PHASES  OF  THE  SYSTEMS-ENGINEERING 
MORPHOLOGY  COMPARED  TO  THE  PHASES  OF  THE 
SYSTEMS- ACQUISITION  PROCESS 


Systems-Engineering 

Time-Dimension 

Phases 1 

Systems-Acquisition 

Life-Cycle 

Phases 

Program  planning 

Mission  analysis  and  development 
of  MENS 

Project  planning 

Conceptual 

Demonstration  and  validation 

System  development 

Full-scale  development 

Production 

Production 

Distribution 

Deployment  or  phase  in 

Operation 

1 .  Operation/support 

2 .  Maintenance/repair 

3.  Modification/retrofit 

Retirement 

Retirement  or  phase  out 

!See  The  Systems -Engineering  Framework  (p.  67  ) . 


COMPARISON  OF  THE  LOGIC-DIMENSION  STEPS  OF  THE  SYSTEMS-ENGINEERING  MORPHOLOGY  WITH  THE 
ACTIVITIES  OF  THE  FOUR  PHASES  OF  THE  SYSTEMS-ACQUISITION  PROCESS 
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contained  in  the  previous  chapter.  The  analysis  identified 
and  defined  a  separate  and  distinct  phase  of  the  systems- 
acquisition  process  prior  to  approval  of  the  MENS — the  Mission 
Analysis  phase;  each  logic-dimension  step  is  identifiable  as 
are  activities  within  the  phase. 

The  knowledge  dimension  encompasses  the  necessary 
professions  and  disciplines,  including  operations-research 
or  management  science,  engineering,  business,  law,  and 
military  or  naval  science. 

The  stage  approach,'*'  used  to  evaluate  and  select 
R&D  projects,  may  also  be  applied  to  systems  acquisition. 

The  stage  approach  is  based  on  the  sequential  nature  of  the 
decision  process  associated  with  development  programs. 

Normally  these  decisions  are  interspersed  with  stages  of 
development  activity  (Brandenburg  and  Langenbert,  1969; 

Baker,  Seigman,  and  Larson,  1971;  Gear  and  Lockett,  1973a; 
and  Gear  and  Lockett,  1973b) . 

Albala  (1975)  pointed  out  that  a  logical  consequence 
of  the  sequential  nature  of  the  decision  process  is  the 
recognition  of  milestones  that  represent  the  completion  of 
different  R&D  phases.  During  the  initial  phases  of  the  R&D 
process,  the  questions  posed  are  primarily  qualitative  in 
nature,  risk  is  very  high,  and  investments  are  relatively  small. 

*"For  the  purposes  of  this  report,  a  stage  is  defined  as  "A  period 
of  the  R&D  effort  limited  by  reasonably  clear  boundaries  determined 
by  specific  objectives  and  scope  of  work  which  is  normally  associated 
with  a  well  defined  decision  point  in  the  overall  systems-acquisition 
effort"  (Albala,  1975) . 
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Toward  the  end  of  the  R&D  process,  the  richest  and  most  accurate 
data  are  available,  the  degree  of  risk  and  uncertainty  is  at 
its  lowest,  and  R&D  costs  are  at  their  most  significant  levels. 
Each  stage  of  the  process  may  be  characterized  by  four  dif¬ 
ferent  factors  (Albala,  1975)  : 

1.  Clear  stage  boundaries; 

2.  a  progressively  decreasing  degree  of  risk  and 
uncertainty; 

3.  a  progressively  increasing  size  of  the  investment; 

4.  an  increasing  length  of  time  required  for  the 
completion  of  the  stage. 

The  four  characteristics  noted  above  can  be  related 
to  the  systems-acquisition  process.  First,  a  meaningful 
analogy  can  be  drawn  between  Albala' s  project  stages  and 
the  phases  of  the  systems-acquisition  process.  Secondly, 
as  previously  discussed,  there  is  a  decreasing  degree  of  risk 
and  uncertainty  and  an  increase  in  R&D-investment  costs  as 
the  program  moves  through  the  phases  of  the  systems-acquisition 
process.  Third,  as  a  general  rule,  the  time  to  complete  each 
phase  is  progressively  longer.  Generalized  curves  depicting 
progressively  increasing  R&D  program  costs  and  progressively 
decreasing  risk  and  uncertainty  as  a  function  of  time  and 
the  systems-acquisition  phases  are  presented  in  Figure  21. 

The  figure  also  portrays  the  decreasing  R&D  costs  at  the 
end  of  the  full-scale  development  phase  and  the  increasing 
production  costs  at  the  beginning  of  the  production  phase. 


Figure  21.  Generalized  Program  Costs  and  Risk/Uncertainty  as  a  Function  of 
the  Systems- Acquisition  Phases.  R&D  and  production  costs  could  differ  by  several 
orders  of  magnitude. 


The  systems-acquisition  process  may  be  cast  into  a 
stage  approach  representation  as  presented  in  Figure  22. 

The  figure  depicts  the  overall  activities  of  systems  acquisi¬ 
tion  in  a  DELTA  chart  format  with  a  stage-approach  structure 
superimposed  upon  it.  The  initial-analysis  stage  is  followed 
by  four  interstage  decision  milestones  alternating  with  the 
R&D  stages  that  represent  the  phases  of  the  acquisition  process 
Each  phase  includes  all  the  steps  of  a  logic  dimension  in 
systems-engineering  as  discussed  in  Chapter  3.  The  figure 
also  includes  a  generalized  breakdown  of  DOD  responsibilities 
for  the  stage  appoarch. 

Consideration  of  the  systems-acquisition  process  as 
a  stage-approach  activity,  with  its  previously  identified 
characteristic,  is  important  to  the  analysis  associated  with 
each  of  the  sequential  decision  milestones.  Distinct 
analysis  models,  adopted  to  the  particular  phase  (or  stage), 
should  be  used  for  evaluation  at  different  decision  points 
rather  than  using  the  same  model  at  all  decision  points 
(Albala,  1975;  Gear  and  Lockett,  1973b;  and  Souder,  1973). 

This  topic  is  addressed  again  in  the  Development  of  the 
Systems  Engineering  Methodology  section  (p.  183  ). 

The  following  observations  on  the  structure  of  the 
systems-acquisition  process  point  the  way  toward  the  develop¬ 
ment  of  a  systems-engineering  methodology: 

1.  That  the  systems-acquisition  process  can  be 
modelled  with  a  systems-engineering  framework; 


167 


2.  that  the  activities  carried  out  during  each  phase 
of  the  systems-acquisition  process  are  repetitive,  with  each 
having  a  major-decision  milestone  that  represents  both  the 
end  of  the  previous  phase  and  the  beginning  of  a  new  phase; 

3.  that  the  systems-acquisition  process  is  multistage 
and  each  phase  is  a  separate  stage  of  the  overall  develop¬ 
ment  from  mission  analysis  to  production  and  deployment; 

4.  that  the  decisions  associated  with  the  systems- 
acquisition  process  are  sequential  and  later  decisions  are 
highly  dependent  on  previous  decisions. 

The  Criteria  and  Organizational  Considerations 
for  Applying  the  Sy stems -Engineering 
Tools  and  Techniques 

The  identification  of  the  criteria  for  applying  the 
systems-engineering  tools  and  techniques  is  important  to 
the  development  of  a  systems-engineering  methodology  for 
the  DOD's  systems  acquisitions.  However,  organizational 
characteristics  must  also  be  considered  to  understand  how  a 
systems-engineering  methodology  can  be  used  for  the  systems- 
acquisition  process. 

A  criterion  is  a  standard,  rule,  or  test  by  which  a 
judgment  of  something  can  be  formed.  The  criteria  for  select¬ 
ing  the  appropriate  analysis  tool  must  incorporate  the  key 
characteristics  of  the  systems-acquisition  program  to  be 
evaluated.  These  key  characteristics  are  identified  when 
the  acquisition  program  is  viewed  in  the  context  of  the 


systems-engineering  morphology  and  as  modelled  by  the  activi¬ 
ties  of  the  stage  approach  (Albala,  1975) .  Key  program 
characteristics  are  noted  below,  with  comments  on  each 
characteristic  and  a  criterion  to  compliment  each  character¬ 
istic  . 

1.  Characteristic :  Data  Availability  for  the  Program 

Comment:  Data,  from  sources  both  internal  and 

external  to  the  organization,  should  be  obtained  with  a  minimum 
expenditure  of  time,  money,  and  effort;  both  the  quantity 
and  quality  of  this  data  should  be  analyzed.  The  data-input- 
to-analysis  models  will  not  drive  the  cost  accounting,  program 
data  management,  and  other  data  collection  functions  within 
the  organization.  Security  requirements  can  restrict  the 
availability  of  data. 

Criterion :  Analysis  methods  must  be  appropriate 
to  the  quantity  and  quality  of  available  data. 

2.  Characteristic :  Stage  of  the  Acquisition  Process 
Comment:  The  levels  of  risk  and  uncertainty  of 

many  aspects  of  the  program  are  governed  by  the  stage  (or 
phase)  of  the  systems-acquisition  program.  The  tradeoffs 
between  quantitative  and  qualitative  data  are  also  dependent 
on  the  stage. 

Criterion :  Analysis  methods  must  be  consistent 
with  features  of  the  specific  stage  of  the  acquisition  process. 
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3.  Characteristic:  Program  Objectives 

Comment:  The  objectives  of  the  program  are  a  key 
in  determining  what  analysis  procedures  to  use.  Programs 
updating  existing  capabilities  are  analyzed  through  minimum 
validation  processes.  Programs  for  entirely  new  systems,  or 
even  revolutionary  technology,  require  a  much  more  thorough 
analysis  effort  (i.e.,  the  Cruise  Missile  Program  or  the 
development  of  VSTOL  aircraft) . 

Criterion :  Analysis  methods  must  be  consistent 
with  the  overall  objectives  of  the  program. 

4.  Characteristic :  A  Step  in  the  Analysis  Effort  for 
the  Program 

Comment:  As  each  activity  is  analyzed,  the  analysis 
passes  through  the  steps  of  the  logic  dimension  of  the  systems- 
engineering  framework.  Each  step  has  specific  tools  and  tech¬ 
niques  identified  with  it  that  should  be  used  during  that 
particular  step.  This  association  of  tools  and  techniques 
with  a  particular  logic-dimension  step  was  recently  documented 
(Sage,  Warfield,  Ihissen,.  et.al.,  1978)  and  is  based  on  the  appli¬ 
cation  of  the  systems-engineering  methodology  to  the  general 
policy  problem. 

Criterion :  Analysis  methods  must  be  appropriate 
to  the  particular  logic -dimen si on  step  being  analyzed  within 
the  systems-engineering  framework. 
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5.  Characteristic :  Extent  of  the  Resources  Required 
for  the  Program 

Comment:  The  systems  life-cycle  costs  and  manpower 
requirements  are  two  examples  of  the  overall  resource  inputs 
to  an  acquisition  program.  The  analysis  effort  for  the  program 
should  be  generally  proportional  to  the  overall  resource 
requirements  for  the  program. 

Criterion :  The  effectiveness  and  reliability  of 
the  analysis  efforts  must  increase  as  the  resource  inputs  to 
the  program  increase. 

The  above  criteria ,  which  must  be  adapted  to  the  speci¬ 
fic  program  being  evaluated ,  are  important  in  selecting  the 
tools  and  techniques  to  be  used  in  a  systems-engineering 
methodology.  The  second  major  factor  in  determining  the 
nature,  type,  and  degree  of  analysis  to  be  performed  is  the 
organizational  environment.  In  the  DOD  context,  several  organi¬ 
zations.  including  the  System  Program  Office,  the  Service 
Headquarters,  and  OSD,  could  evaluate  an  acquisition  program. 

Several  articles  have  been  written  on  different 
aspects  of  the  problem  of  using  analysis  procedures  in 
decision  making.  Hast  and  Rosenzweig  (1974)  considered  the 
behavioral  aspects.  Souder  (1967)  and  Radnor,  Rubenstein,  and 
Tansik  (1970)  presented  implementation  procedures  for  opera¬ 
tions  and  research  in  the  R&D  environments  of  government  and 
business.  The  complexity  of  decision  making  in  an  R&D  environ¬ 
ment  was  also  discussed  by  Hill  and  Ollila  (1978);  management 
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structure  concerns  for  dealing  with  this  complexity  was 
addressed  by  Greenblott  and  Hung  (1970).  Polishuk  (1976) 
specifically  covered  problems  related  to  systems-engineering 
(interdisciplinary)  policy  research  and  management  in  the 
federal  government. 

Although  these  articles  have  provided  a  general  frame 
of  reference  for  using  analysis  procedures  in  decision  making, 
they  did  not  deal  with  the  organizational  aspects  of  selecting 
procedures.  Based  on  the  author's  experience  in  Defense 
R&D,  the  following  several  organizational  considerations 
affect  the  nature,  type,  and  degree  of  an  analysis  selected 
to  support  decision  making: 

1.  Predisposition  of  the  organization  to  use  analysis 
procedures .  An  organization's  willingness  to  use  systems 
engineering  and  management-science  techniques  is  dependent 
on  the  history  of  use  in  that  organization.  These  can  be 
classified  as  follows:  those  that  have  successfully  used  the 
procedures,  those  that  have  used  them  with  little  or  no 
success,  and  those  that  have  not  used  them  at  all.  The 
latter  two  are  going  to  be  unlikely  to  use  any  analysis 
procedures  regardless  of  the  demands  of  the  specific  decision¬ 
making  situation.  They  may  be  reoriented  to  accept  analysis 
support,  but  it  will  take  time  and  patience  on  the  part  of 
the  analysts. 
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2.  Position  of  the  analysis  unit  within  the  organiza¬ 
tion.  As  a  general  rule,  the  closer  the  analysis  unit  is  in 
the  organizational  structure  to  the  actual  decision  maker  or 
policy  maker,  the  greater  the  impact  of  the  analyses.  Filters 
between  the  managers  and  the  analysts  dilute  the  analysis 
effort  proportionately.  Organizational  proximity  will  not 
guarantee  the  quality  of  the  analysis,  but  it  will  enhance 
their  leverage. 

3.  Background  and  experience  of  the  decision  makers. 

Their  educational  and  professional  backgrounds  influence  the 
degree  to  which  the  decision  makers  accept  analysis  procedures. 
Those  with  some  technical  background  are  usually  willing  to 
consider  at  least  some  sort  of  analysis  assistance.  Analysts 
must  know  the  backgrounds  of  their  decision  makers. 

4.  Availability  of  decision  makers.  Time  is  a 
critical  factor  in  any  manager's  professional  day.  Therefore, 
an  analyst  must  adapt  the  analysis  procedure  to  the  available 
time  of  his  decision  maker.  Analysis  procedures  that  require 
excessive  managerial  time  are  not  likely  to  be  used. 

5.  Mechanism  used  to  transfer  the  results  of  the 
analysis  to  the  decision  makers.  The  procedures  associated 
with  the  decision  process  are  important  in  determining  what 
type  of  analysis  procedures  should  be  used.  The  one-shot 
briefing  in  which  only  the  results  of  the  analysis  are  presented 
cannot  accomodate  complex  analysis  procedures?  those  analyses 
which  cannot  be  grasped  by  the  manager  in  a  short,  one-half 


hour  briefing  will  not  be  used.  On  the  other  hand,  if  pre¬ 
briefings,  informal  conferences,  or  working  groups  are  held 
first  then  more  complex  analyses  may  be  used  and  acceptance 
is  increased. 

6.  Criticality  of  the  decision  to  the  organization. 

The  depth,  extent,  and  nature  of  the  analysis  effort  should 
be  related  to  how  important  a  particular  decision  is  to  the 
organization.  Although  the  analysts  should  be  aware  of  the 
decision  makers'  values,  they  must  also  be  sensitive  to  the 
organization's  value  systems.  Resources  assigned  to  a  parti¬ 
cular  program  are  not  the  only  criteria  for  determining 
the  importance  of  that  program.  Traditional  mission-oriented 
needs,  such  as,  tactical  aircraft  development  for  the  Air 
Force,  and  external  political  influences  are  two  other  criteria 
for  measuring  a  program’s  importance. 

7.  Availability  of  resources  for  analysis.  The 
availability  of  resources  for  analysis,  including  qualified 
analysts,  technical/administrative  support  personnel,  and 
computer  facilities,  is  a  key  factor  in  determining  the  type 
and  extent  of  analysis.  Managers  for  analysis  units  should 
be  keenly  aware  of  the  planning  and  budgets  of  the  organiza¬ 
tion  so  that  ample  provisions  are  made  for  future  analysis 
efforts. 

8.  Mode  of  organizational  operation.  In  this  situa¬ 
tion,  the  type  of  analysis  to  be  performed  is  related  to  the 
type  of  policy  roles  being  assumed  by  the  policy  maker  within 
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the  organization.  Polishuk  (1976)  defined  three  policy 
roles:  (a)  "Downstream  policy  analysis":  the  policy 

maker  makes  an  arbitrary  decision  about  a  policy  matter  and 
the  organization  is  placed  in  an  advocacy  role  to  support 
this  decision;  (b)  "Reactive  policy  analysis":  external 
forces  encroach  upon  the  policy  makers'  areas  of  authority 
and  the  organization  assumes  a  defensive  role;  (c)  "Upstream 
policy  analysis":  the  policy  maker,  in  attempting  to  be  a 
leader  and  manager  exercises  initiative  instead  of  merely 
reacting  to  external  forces.  The  policy  maker  wants  to 
evaluate  alternatives  and  desires  comprehensive  information 
about  the  area  over  which  the  position  has  management 
responsibility.  The  inexperienced  analyst  often  anticipates 
working  only  in  an  organization  that  is  operating  in  the 
third  policy  role.  When  the  organization  is  in  this  policy 
role,  the  analyst  is  free  to  select  the  analysis  procedures 
technically  best  fitted  to  the  problem  under  consideration. 
However,  all  too  often  the  organization  is  operating  in  one 
of  the  first  two  policy  roles  causing  the  analyst  to  use 
analysis  procedures  selected  by  organizational  rather  than 
technical  criteria. 

The  organizational  considerations  developed  above  are 
addressed  again  in  the  section  dealing  with  the  implementa¬ 
tion  of  the  systems-engineering  methodology. 


The  Emphasis  of  the  Sy stems -Engineering  Methodology 

Those  areas  of  the  overall  Defense  systems-acquisition 
process  that  should  be  emphasized  in  developing  a  systems- 
engineering  methodology  are  identified  below  from  several 
perspectives.  First,  the  perceived  value  system  of  the 
DSARC/ (S) SARC  panels  are  examined  to  see  how  a  systems- 
engineering  methodology  could  assist  the  panel  members  in 
obtaining  their  objectives.  Second,  the  problem  definitions 
(see  Chapter  2,  p.  36)  are  briefly  reviewed  to  show  the  scope 
and  focus  of  the  methodology.  Third,  the  structure  and  nature 
of  the  systems-acquisition  process  (see  The  Nature  and  Structure 
of  the  Defense  Systems-Acauisition  Process ,  p.  144)  are  used  to 
identify  the  specific  problems  for  analysis  that  are  signifi¬ 
cantly  important  to  management  and  particularly  receptive  to 
analysis  efforts. 

The  perceived  value  systems  for  the  DSARC  and  the 
(S) SARC  panels  are  presented  by  the  objectives  tree  of 
Figure  18  (p.  137).  This  figure  shows  that  objectives  3.1 
through  3.5  relate  to  considerations  for  DSARC  dealing, 
respectively,  with  congressional  relations,  the  direction  of 
the  Executive  Branch,  the  Defense  industry,  the  internal  DOD 
systems-program  management,  and  the  international  security. 

These  objectives  3.1  through  3.5  deal  with  problem 
considerations  that  must  be  integrated  into  the  systems- 
acquisition  decision  process.  However,  these  considerations 


176 


are  difficult  to  address  through  systems-engineering  because 
of  the  DOD  perspective  used  in  this  disseration.  Many  of 
these  problems  can  be  solved  by  nonrigorous  management 
techniques  and  behavioral  approaches  based  on  sound 
judgment,  wisdom,  and  professional  experience.  In  a 
recent  article,  Freund  (1979)  advocated  the  necessity  to 
rely  ultimately  on  judgment  and  experience  in  addressing 
complex  issues.  He  further  stated  that  analysis  and 
research  can  only  be  used  to  a  certain  extent  in  addressing 
problems,  then  judgment  must  be  called  upon  to  resolve  the 
issue . 

The  area  of  consideration  that  is  fertile  ground 
for  applying  systems-engineering  analyses  is  that  repre¬ 
sented  by  objective  3.6  in  Figure  18. — "to  develop  the 
best  overall  program  of  systems-acquisitions  considering 
Defense-related  constraints."  The  subobjectives  under 
objective  3.6  deal  with  program  selection  and  evaluation, 
criteria  development,  key-decision  points,  standardization 
requirements,  and  the  development  of  supporting  program 
documentation.  These  subobjectives  point  the  way  toward 
areas  of  emphasis  for  the  systems-engineering  methodology. 

The  problem  definitions  in  Chapter  2  (p.  36  ) 
established  the  scope  and  focus  of  the  problem  to  be 
addressed  by  the  systems-engineering  methodology.  The 
problems  are  specifically  oriented  toward  the  DOD 


environment  and  the  major-decision  milestones  associated 
with  a  systems-acquisition  process.  The  problem  must 
be  formulated  within  the  existing  policy  framework  as 
stipulated  by  OMB  A-109  and  DOD  5000.1/5000.2  (0MB, 

1976a;  Defense,  1977a,  1977b)  but  new  policy  initiatives 
should  be  addressed  when  appropriate.  The  methodology 
concentrates  on  the  systems-acquisition  decision  process 
at  the  service-headquarters  and  OSD  levels.  The  emphasis 
of  these  considerations  in  relation  to  the  development 
of  a  methodology  are  summarized  in  the  research  problem 
statement  for  this  effort:  To  determine  how  the  Department 
of  Defense  systems -acquisition  process  may  be  improved  using  systems 
engineering .  The  emphasis  of  this  study  will  be  directed 
toward  the  four  major-decision  milestones  (0,  I,  II,  and 
III),  the  operation  of  the  DSARC  and  (S)SARC,  and  the  use 
of  the  Decision  Coordination  Paper  (DCP)  and  Mission 
Element  Need  Statement  (MENS)  within  the  overall  DOD- 
policy  framework. 

The  problem  definition  and  the  perceived  value 
system  operative  within  the  OSD  and  the  Service  Headquarters 
(specifically  within  the  DSARC  and  (S)SARCs)  point  the 
direction  toward  the  general  problem  areas  where  a  systems- 
engineering  methodology  can  be  applied.  However,  it  is 
the  nature  and  structure  of  the  systems-acquisition  process 
(p.  144)  that  identify  the  specific  problem  elements  to 
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which  the  analysis  procedures  of  the  systems-engineering 
methodology  should  be  applied.  These  observations  are 
reviewed  below  and  specific  problem  elements  are  identified 
for  analysis  within  the  systems-engineering  methodology. 

In  reviewing  the  observations  on  the  nature  of 
the  systems-acquisition  process,  the  first  major  area  of 
emphasis  to  be  identified  is  the  development  of  the  MENS. 

This  document  presents  the  results  of  the  initial  problem- 
definition  step  of  the  systems-acquisition  process.  It 
is  also  used  as  the  basis  for  approving  new  starts  and  is, 
therefore,  a  critical  document.  Because  of  the  importance 
of  this  document  and  of  the  clarity  of  its  contents  (see 
Defense,  1978b),  the  format  of  the  MENS  must  be  improved. 
Therefore,  the  first  task  of  developing  the  systems- 
engineering  methodology  is  to  outline  a  more  rigorous  buy 
yet  flexible  format  for  the  MENS. 

This  initial  task  in  developing  the  methodology 
involves  two  of  the  recommended  DOD  systems-acquisition-policy 
initiatives  covered  in  the  observations  on  the  nature  of  the 
systems-acquisition  process  (p.144).  Specifically,  it  is 
related  to  the  recommended  policy  initiatives  on  formalizing 
the  de  faato  mission-analysis  phase  into  an  official  phase 
of  systems  acquisition.  This  task  also  relates  to  the  policy 
initiative  for  a  more  clearly  defined  problem-definition 
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approach  to  the  MENS.  Both  these  policy  initiatives  are 
specifically  addressed  in  Development  of  the  Systems- 
Engineering  Methodology  ,  (p.183). 

Two  of  the  other  recommended  policy  initiatives  pre¬ 
sented  in  the  observations  on  the  nature  of  the  systems- 
acquisition  process  (p.144)  are:  (1)  the  adoption  of  a 
more  flexible  overall  approach  for  the  process  and  (2)  the 
integration  of  the  OMB  A-109  (1976a)  policies  into  the  pro¬ 
gramming  requirements  of  the  PPBS  and  the  congressional  budget 
cycle.  Since  these  issues  have  been  considered  by  OSD  and  are 
included  in  the  revised  versions  (December  18,  1978)  of  DOD 
Directives  5000.1  and  5000.2,  they  are  not  addressed  again 
in  this  paper. 

The  second  major  area  of  emphasis  for  the  methodology 
is  that  of  trading  off  the  various  potential  programs  early 
in  the  acquisition  process.  A  very  subtle  type  of  tradeoff 
currently  takes  place  at  the  Milestone  0  decision  point.  The 
mission-element  need  becomes  a  yardstick  for  measuring 
requirements;  the  existing  capability  is  measured  against 
this  yardstick.  If  the  capability  falls  short  of  the 
yardstick,  then,  almost  by  definition,  some  new  start  is 
required  whether  it  is  a  completely  new  acquisition  program 
or  a  major  modification. 

The  problem,  of  course,  is  that  the  programs  are 
not  traded  off,  one  against  the  other,  until  further  into 
the  systems-acquisition  cycle  when  funding  requirements  are 
more  readily  identifiable.  They  are  traded  off  when 


180 


funds  are  constrained  and  with  little  management  control. 

Therefore,  the  second  task  of  developing  the  systems- 
engineering  methodology  is  to  establish  a  procedure  for 
evaluating  and  selecting  the  programs  to  be  approved  for 
official  program  initiation.  This  is  the  final  policy 
initiative  presented  in  the  observations  on  the  nature  of 
the  systems-acquisition  process  (p.144).  The  policy  implica¬ 
tions  of  the  selection  and  evaluation  procedures  of  this  task 
are  addressed  in  th e  Development  of  the  Systems-Engineering  Methodology  (p.183)  . 

The  final  major  area  of  emphasis  for  the  methodology 
as  indicated  by  the  observation  on  the  nature  and  structure 
of  the  process  is  that  of  addressing  the  major-decision 
Milestones  I,  II,  and  III.  Two  types  of  decisions  are  con¬ 
sidered  at  Milestones  I,  II,  and  III  that  could  be  analyzed 
with  systems-engineering  procedures.  The  first  type  deals 
with  evaluating  and  selecting  alternative-candidate  solutions 
to  be  advanced  to  the  next  phase  of  the  acquisition  process. 

This  decision  begins  in  the  systems-program  office  and  con¬ 
tinues  through  the  DSARC/ (S) SARC  process  to  the  OSD  level. 

The  second  type  of  decision  is  the  go/no-go  determination  for 
an  overall  program  to  proceed  to  the  next  phase.  This  decision 
is  usually  forwarded  through  the  (S)SARC  and  DSARC  with  recom¬ 
mendations  to  the  SERVSEC  and  SECDEF,  respectively.  The  first 
type  of  decision  provides  input  to  the  second  decision. 

The  relationship  of  the  two  types  of  decisions  to 
the  major-decision  milestones  and  their  importance  to  the 
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overall  systems-acquisition  process  are  given  in  Table  8. 

This  shows  the  similarity  between  major  Milestone  I  and  II 
where  both  types  of  decisions  must  be  considered.  When  the 
nature  of  the  systems-acquisition  process  is  considered  as  a 
stage-approach  problem,  the  decisions  at  Milestone  I  and  II 
are  made  when  there  is  still  risk  and  uncertainty  in  the 
program  and  during  the  transition  from  qualitative  to  quanti¬ 
tative  data.  The  decision  situations  at  Milestones  I  and  II 
may  be  contrasted  to  the  situation  at  Milestone  III.  At 
the  latter,  only  one  type  of  decision  must  be  made — the 
go/no-go  determination  for  production.  This  decision  is 
also  made  at  the  end  of  the  development  effort  when  risk 
and  uncertainty  are  normally  lowest  and  quantitative  data  is 
most  available. 

Considering  the  differences,  there  are  really  two 
distinct  areas  of  analysis  related  to  the  major-decision 
milestones :  the  analyses  required  for  Milestones  I  and  II, 
and  the  analysis  required  for  Milestone  III. 

Therefore,  four  separate  areas  have  been  identified 
for  emphasis  in  the  systems-engineering  methodology.  Each 
area  of  emphasis  has  an  analysis  task  associated  with  it  that 
is  addressed  in  the  development  of  the  methodology.  These 
analysis  tasks  are: 

1.  To  develop  a  more  rigorous  and  well-defined  MENS; 

2.  to  develop  an  analysis  procudure  to  trade  off  newly 
identified  programs  (as  represented  by  their  MENS)  early  in 
the  systems-acquisition  process. 
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TABLE  8 


TYPES  OF  DECISIONS  CONSIDERED  AT  DECISION 
MILESTONES  I,  II,  AND  III 


Decisions 

Major-Decision 

To  Evaluate  and  Select 
Alternative  Candidate 
Solutions  to  Advance 

Go/No-Go 

Advance 

to 

Determination  to 
Overall  Program 
Next  Phase 

Milestone 

to  Next  Phase 

I 

Approval  to  proceed 
to  demonstration- 
and-validation 
phase 


Most  critical  because 
of  number  of  alter¬ 
natives  to  be 
considered 


Less  critical  than 
Milestone  III;  pro¬ 
jected  fund  require¬ 
ments  are  lower, 
production  decision 
is  not  made 


II 

Approval  for  produc¬ 
tion  and  deployment 


III 

Approval  for  produc¬ 
tion  and  deployment 


Less  critical  than 
Milestone  I;  less 
alternatives  are 
considered 


Most  critical;  approval 
to  proceed  should  not 
be  granted  unless 
production  is  very 
probable 

Almost  as  critical  as 
Milestone  II;  final 
review  before  funds 
for  production  and 
deployment  are 
committed 


Not  required 
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3.  to  develop  an  analysis  procedure  to  support  the 
DSARC/ (S) SARC  decision  process  at  major-decision  Milestones 
I  and  I I ; 

4.  to  develop  an  analysis  procedure  to  support  the 
DSARC/ (S) SARC  decision  process  at  major-decision  Milestone  III. 

Development  of  the  Sy stems-Engineering  Methodology 

Methodology  has  been  defined  (Sage,  1977)  as  an  open 
set  of  procedures  that  provides  the  means  for  solving 
problems.  This  definition  can  include  three  separate  com¬ 
ponents  related  to  developing  and  using  the  methodology. 

First,  the  problems  to  be  addressed  by  the  methodology 
must  be  identified.  This  requires  studying  the  overall 
systems  problem  with  all  its  inherent  policy  arid  organizational 
considerations  and  isolating  those  specific  problem  areas 
where  the  application  of  analysis  procedures  would  be  most 
appropriate  and  beneficial.  Second,  the  types  of  systems- 
engineering  tools  and  techniques  must  be  determined  for  the 
specific  problem  areas.  Third,  the  most  appropriate  systems- 
engineering  analysis  procedures  must  be  selected  and  initiated. 

The  accomplishment  of  these  three  requirements  pre¬ 
supposes  the  establishment  of  several  systems-engineering 
teams  to  develop  and  implement  the  methodology.  These  teams 
must  be  located  within  OSD  and  at  the  headquarters  of  the 
several  military  services.  A  detailed  discussion  of  the 


functions  and  characteristics  of  these  projected  systems- 
engineering  teams  is  presented  in  the  next  section. 

The  previous  section.  The  Emphasis  of  the  Systems- 
Engineering  Methodology ,  (p.1751,  identifies  specific  problem 

areas  of  the  overall  systems  problem  for  the  application  of 
analysis  procedures.  The  actual  means  by  which  this  was  carried 
out  is  graphically  portrayed  in  the  DELTA  chart  of  Figure  23. 
This  chart  includes  the  analysis  efforts  presented  in  several 
sections  of  this  dissertation.  However,  the  chart  also  shows 
a  general  set  of  procedures  that  can  be  applied  to  the 
study  of  complex  decision-making  processes  in  large  organiza¬ 
tions.  These  procedures  follow  the  logic-dimension  steps 
of  the  systems -engineering  morphology  and  result  in  the 
identif ication  of  the  specific  analysis  tasks  that  need  to 
be  performed.  The  chart  effectively  presents  a  graphical 
algorithm  for  the  selection  of  the  analysis  tasks  of  the 
methodology.  One  feedback  loop  is  included  in  the  chart  to 
indicate  that  the  need  to  revise  the  organization's  policy 
and  procedures  may  be  identified  through  the  analysis 
associated  with  developing  the  systems-engineering  methodology. 
In  a  more  detailed  DELTA  chart,  multi-f eedback  loops  could  be 
used  for  this  set  of  procedures.  The  procedures  of  Figure  23 
assume  a  close  relationship  between  the  analyst  and  the 
organization. 

The  second  requirement  for  developing  the  methodology 
is  to  determine  the  appropriate  systems-engineering  tools 


and  techniques  to  be  applied  to  each  previously  identified 
problem-analysis  area.  The  selection  criteria  associated  with 
the  program  characteristics  and  the  organizational  considera¬ 
tions  for  selecting  analysis  procedures  (see  The  Nature  of 
the  defense  Systems  Acquisition  Process ,  p.  144)  are  used  in 
determining  which  tools  and  techniques  should  be  used  in 
addressing  a  specific  analysis  task. 

The  systems-acquisition  process,  as  modelled  by  the 
systems-engineering  morphology  (Chapter  3,  p.62),  is  of 
particular  interest  in  this  section.  The  logic  dimension 
of  the  systems-engineering  morphology  is  viewed  more  generally 
to  assist  in  the  current  research.  Sage,  Warfield,  Thissen,  et  aZ.,(1978) 
indicated  that  it  is  convenient  to  aggregate  the  seven  steps 
of  the  logic  dimension  of  the  morphology  into  three  more 
general  steps.  First,  the  problem-definition,  value-system- 
design,  and  system-synthesis  steps  may  be  aggregated  into 
a  single  step  called  input  description  and  specification. 

The  systems-analysis-and-modelling  step  and  the  optimization 
of  each  alternative  step  can  be  combined  into  a  single  step 
called  impact  assessment .  The  decision-making  and  planning- 
for-action  steps  can  also  be  collected  in  a  single  step 
called  output  specification  and  interpretation .  Figure  24 
illustrates  the  iterative  feedback  nature  of  these  three 
generalized  steps.  Figure  25  illustrates  the  hierarchical 
structure  associated  with  the  systems  engineering  in  accordance 
with  the  contextual  relation  is  a  component  of.  Each  analysis 


Figure  24.  Iterative  Nature  of  the  Three  Primary  Steps  in  the  Systems- 
Engineering  Processes  ( SOURCE :  Sage,  Warfield,  Thissen,  et  al . ,  1978). 


Figure  25.  Hierarchical  Structure  of  the  Steps  of  Systems  Engineering 
Source:  Sage,  Warfield,  Thissen,  et  al..  1978). 
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procedure  addressed  in  this  section  is  associated  with  one 
of  the  three  generalized  systems-engineering  steps  discussed 
above . 

The  third  requirement  for  developing  and  using  the 
systems-engineering  methodology  is  the  need  to  implement  the 
methodology.  This  requirement  will  be  addressed  in  the  next 
section,  Implementation  of  the  Methodology ,  (p.  237). 

In  the  previous  section,  four  specific  problem  areas 
of  the  systems-acquisition  decision  process  were  selected 
for  emphasis  in  the  systems-engineering  methodology.  Each 
specific  problem  area  must  be  addressed  by  a  separate  task 
to  develop  appropriate  analysis  procedures.  The  four  tasks 
developed  for  the  analysis  procedures  are  outlined  below. 

Development  of  Analysis  Procedures 

Task  #1:  To  Develop  a  Move  Flexible , 

Better-Defined  Documentation 
Process  for  the  MENS 

Introduction 

The  MENS  is  intended  to  document  the  operational 
need  for  a  new  or  improved  mission  capability.  A  mission 
need  arises  from  a  projected  deficiency  (or  obsolescence) 
in  an  existing  system,  a  technological  opportunity,  or  an 
opportunity  to  reduce  operating  cost.  There  is  not  anything 
particularly  new  about  the  concept  of  identifying,  defining, 
and  documenting  operational  requirements  based  on  the  missions 
of  the  different  services.  The  concept  of  relating  R&D 
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programs  sponsored  by  the  military  services  to  future  military- 
mission  requirements  was  described  by  Clarke  (1967).  Jordan 
(1974)  discussed  the  operational-requirements  determination 
process  for  major  Naval  weapon  systems.  He  discussed  such 
terms  as  general  operational  requirement  (GOR) ,  specific 
operational  requirement  (SOR) ,  and  tentative  specific  opera¬ 
tional  requirement  (TSOR) ,  all  of  which  described  naval- 
operational  requirements  during  the  late  1960 's  and  early 
1970's. 

The  significant  changes  in  the  overall  development 
of  operation  requirements  with  the  advent  of  the  OMB  A-109 
(1976a)  and  the  MENS  (Defense,  1978b)  have  been  in  the  pro¬ 
cessing  and  approval  of  the  requirements  document.  Previously, 
this  document  was  approved  by  the  separate  services,  now  the 
MENS  must  be  approved  by  the  SECDEF . 

Because  the  requirements  for  the  MENS  are  not  as 
clearly  defined  as  they  could  be,  the  characteristics  of  the 
mission-element  need  are  not  being  documented  in  the  most 
efficient  possible  way.  This  task  will  present  an  analysis 
procedure  for  a  more  clearly  defined  mission-element  need. 

Relationship  to  the  generalized 
systems-engineering  steps 

The  material  presented  in  Chapter  3  (p.62)  shows  that 
developing  and  documenting  the  MENS  is  the  first  general  step 
(input  description  and  specification)  of  the  systems- 
engineering  framework  for  the  mission-analysis 
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(program-planning)  phase  of  the  systems-acquisition  process. 

The  MENS  is  the  foundation  document  for  the  possible  approval 
and  implementation  of  a  major  systems-acquisition  program. 

It  is  the  basis  for  the  major-decision  Milestone  0  and  also 
must  be  reviewed  and  updated  at  each  subsequent  major-decision 
milestone  to  insure  that  the  operational  need  is  still  valid. 

Analysis-procedure  selection 
considerations 

Based  on  the  program  characteristics,  there  are  two 
important  considerations,  data  availability  and  program 
objective.  Data  availability  is  limited  in  quantity  and 
quality;  the  program  objective  at  this  point  is  to  clearly 
define  the  problem  in  an  integrated,  identifiable,  and  visible 
manner.  The  availability  of  resources  for  analysis  is  par¬ 
ticularly  important  to  this  task.  Most  of  the  MENS  will  be 
developed  by  the  service  and  its  headquarters.  There  should 
be  sufficient  analysis  resources  to  accomplish  this  step 
thoroughly. 

Analysis  Procedure 

As  noted  in  The  Sy stems- Engineering  Framework  and  the 

DOD  Sy  stems- Acquisition  Process  (p.  100)  ,  the  development  and 
review  of  MENS  during  the  mission-analysis  phase  of  the  systems- 
acquisition  process  represents  the  program-planning  phase  of 
the  systems-engineering  framework.  In  particular,  DOD 
Directive  5000.2  requires  that  the  MENS  address  such  areas 
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as  mission  area  (objectives),  needs,  existing  capabilities, 
constraints,  assessment  of  impacts  of  not  acquiring  the 
projected  capability,  and  plans  to  identify  and  explore 
competitive  alternative  systems.  These  requirements  of 
the  MENS  are  very  similar  to  the  products  that  would  be 
anticipated  from  carrying  out  the  first  three  steps  of  the 
logic  dimension  of  the  systems-engineering- framework : 
problem  definition,  value  systems  design,  and  systems 
synthesis . 

Hill  and  Warfield  (1972)  developed  a  technique  by 
which  the  first  three  steps  of  the  program-planning  phase 
of  systems  engineering  could  be  accomplished  for  complex 
programs  with  multiple  relationships.  This  technique  was 
called  Unified  Program  Planning  and  it  set  forth  a  graphical 
approach  by  which  the  steps  of  the  program  planning  phase 
could  be  addressed  as  a  connected  set.  Because  of  the 
complexity  of  the  Defense  programs  that  are  evaluated  by  the 
systems-acquisition  process.  Unified  Program  Planning  was 
chosen  as  the  means  to  carry  out  the  mission-analysis  (program 
planning)  phase  of  the  process  and  thereby  develop  the  MENS. 
Interpretive  Structural  Modeling  (Warfield,  1976)  would 
extend  the  capabilities  of  Unified  Program  Planning  and 
would  provide  a  more  efficient  and  explicit  method  of 
handling  the  generated  data. 
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Results  and  anticipated  benefits 

Since  the  various  areas  of  concern  addressed  by  the 
MENS  and  outlined  in  DOb  Directive  5000.2  (Defense,  1977b) 
are  the  goals  of  this  analysis  task,  the  threat,  existing  and 
planned  capabilities,  and  the  mission  will  all  be  addressed. 
However,  these  and  other  considerations  will  be  studied 
more  comprehensively  than  envisioned  by  the  current  DOD 
directives.  The  products  for  the  three  steps  are 
listed  from  Hill  and  Warfield  (1972): 

1.  Problem  Definition  Products. 

a.  A  well-conceived  title  for  the  problem  or  issue. 

b.  A  descriptive  scenario  explaining  the  nature  of 
the  problem  and  how  it  came  to  be  a  problem, 
presenting  as  much  history  and  data  as  can  be 
prepared  with  available  resources. 

c.  An  understanding  of  what  disciplines  or  professions 
are  relevant  to  the  problem. 

d.  An  assessment  of  scope. 

e.  A  determination  of  the  Defense  sectors  involved. 

f.  An  identification  of  the  people  to  be  involved 
in  the  problem-solving  situation. 

g.  An  identification  of  need. 

h.  An  identification  of  alterables  (those  elements 
in  the  system  that  are  subject  to  change.) 

i.  An  identification  of  major  constraints. 

j.  A  breakdown  of  the  problem  into  relevant  elements. 

k.  Some  isolation  of  the  subjective  elements  of  the 
problem. 

2.  Value  System  Design  Products. 

a.  A  set  of  objectives  for  the  program  arranged 
in  a  hierarchical  structure. 

b.  A  set  of  graphical  linkages  which  would  relate  the 
objectives  to  needs,  constraints,  and  alterables. 
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c.  A  set  of  measures  defined  on  the  objectives  by 
which  to  measure  the  attainment  of  objectives. 

3.  System  Synthesis  Products. 

a.  A  graphical  representation  of  the  relationships 
between  the  planned  activities  and  the  program 
activities . 

b.  A  graphical  representation  of  the  interactions 
between  the  planned  activities  and  the  program 
constraints. 

c.  A  measurement  system  required  for  relating  the 
progress  of  the  activities  to  the  attainment  of 
the  objectives. 

Results  from  this  analysis  procedure  would  more  clearly 
define  a  mission-element  need  with  well-identified  and  highly 
visible  problem-element  interactions  and  interrelationships. 


Capsule  description  of  the 
analysis  procedure 

Normally  this  general  step  of  the  system-engineering 
morphology  requires  a  group  to  analyze  the  overall  mission- 
element  need  and  any  projected  associated  program.  This  group 
would  consider  the  holistic  problem  in  a  mode  of  thinking  called 
outsc oping .  Hill  and  Warfield  (1972)  defined  outscoping  as 

A  deliberate  group  attempt  to  embed  the  problem  or  issue 
in  the  next  larger  problem  or  issue  iteratively  in  order  to 
expand  the  scope  until  the  problem  or  issue  is  seen  in  an 
encompassing  context. 

When  this  context  is  reached,  the  group  can  identify  the 
various  elements  of  the  several  problem  sets  and  the  inter¬ 
actions  among  them. 

The  problem  arises  as  to  how  to  develop,  record, 
and  communicate  the  products  of  a  group.  Graphics  may  be  used 
for  this  purpose,  particularly  trees  and  matrices,  to  unify 
the  output  products  enumerated  above. 
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In  the  problem-definition  step,  the  various  relation¬ 
ships  between  constraints,  alterables,  needs,  and  Defense 
sectors  must  be  addressed.  This  can  be  done  by  self¬ 
interaction  (Figure  26)  and  cross-interaction  (Figure  27) 
matrices,  which  show  interactions  between  elements  of  the 
same  set  or  interactions  between  elements  of  differing  sets, 
respectively.  Levels  of  interaction  can  also  be  shown  and 
the  overall  relationship  of  the  several  sets  can  be  portrayed 
as  shown  in  Figure  28 .  This  means  of  presentation  allows 
the  following  products  of  the  problem-definition  step  to  be 
graphically  portrayed  at  one  time. 

1.  The  defense  sectors  involved; 

2.  the  identified  needs; 

3.  the  identified  alterables; 

4.  the  identified  major  constraints; 

5.  a  description  of  the  interactions  among  the 
relevant  elements  of  the  problem. 

In  the  value-system  design  phase,  objectives  trees 
similar  to  those  used  elsewhere  in  this  dissertation  should 
be  used  to  portray  the  hierarchical  relationships  among  the 
objectives  for  the  program  or  system.  This  approach  would  be 
useful  in  placing  a  mission-element  need  in  the  context  of 
its  mission  area.  A  specific  syntax  should  be  used  to  form 
an  objective;  infinitive  verb  +  object  word  (or  phrase)  + 
constraints .  The  objectives  tree  is  formed  by  carrying  out 
the  following  rules  (Hill  and  Warfield,  1972): 


Ob?«ctiv«s 
Interaction 
Matrix  / 


CONSTRAINTS 


ALTERABLES 


NEEDS 


Figure  28.  Interaction  of  Objectives  with  Constraints 
Alterables ,  Needs,  and  Defense  Sectors  (adapted  from  Hill  and 
Warfield,  1972)  . 
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1.  Write  each  objective  within  a  box  to  form  the  vortex 
of  a  tree. 

2.  Connect  two  boxes  containing  two  objectives,  if  achieve¬ 
ment  of  one  of  the  objectives  contributes  directly  to 
achievement  of  the  other. 

An  example  of  an  objectives  tree  is  shown  in  Figure 
4  (p.  45  ) .  A  cross-interaction  matrix  can  be  used  to  show 

the  interrelationships  between  the  objectives  measures  and  the 
objectives  as  shown  in  Figure  29.  The  products  of  the  system 
synthesis  may  be  represented  by  interaction  and  cross¬ 
interaction  matrices  as  shown  in  Figure  30. 

Input  requirements 

The  input  to  this  analysis  process  comes  in  the 
form  of  thoughts,  concepts,  ideas,  and  information  about  the 
various  problem  elements.  The  information  is  associated  with 
the  MENS  components,  such  as,  the  enemy  threat  and  the  per¬ 
formance  characteristics  of  existing  capabilities.  However, 
the  thoughts,  concepts,  and  ideas  must  come  from  the 
personnel  involved  in  the  problem-definition  effort. 

Several  methods,  including  brain  writing,  brain 
storming,  and  the  DELPHI  method,  could  be  used  to  generate 
the  required  problem  elements.  A  procedure  particularly 
applicable  to  the  DOD  environment  is  the  Nominal  Group 
Process  (NGP) .  This  is  a  structured  group  meeting  where  six 
to  ten  people  sit  around  a  table  in  full  view  of  one  another. 
Each  individual  writes  his  own  ideas  on  a  pad  without  consulting 
any  other  member  of  the  group.  After  approximately  five  to 


OBJECTIVES 


Figure  29.  Interrelationships  Between  the  Objec 
tives  Measures  and  the  Objectives  (adapted  from  Hill  and 
Warfield,  1972). 


Activities  features 


Objectives 


Constraints 


Figure  30.  Linkages  of  the  System-Synthesis  Phase 
Represented  by  Interaction  and  Cross-Interaction  Matrices 
(adapted  from  Hill  and  Warfield,  1972) . 
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fifteen  minutes,  each  person,  in  his  turn,  presents  a  single 
idea  from  his  personal  list.  The  ideas  are  recorded  on  a  flip 
chart;  this  process  continues  until  each  member  has  divulged 
his  list  of  ideas.  Next,  the  ideas  Listed  on  the  flip  chart 
are  clarified  and  expanded.  After  a  thorough  discussion,  each 

A  *  \ 

member  of  the  group  selects  his  own  choice  of  ideas  indicating 
his  order  of  priority  (Delbecq,  Van  de  Van,  and  Gustafson, 

1976)  . 

Operational  and  analysis 
assumptions 

From  the  operational  standpoint  of  DOD,  it  is  assumed 
that  an  emerging  mission-element  need  requires  a  more  clearly 
identified,  properly  developed,  and  meaningful  document.  It 
is  also  assumed  that  there  are  service  personnel  who  have  a 
qualitative  understanding  of  the  mission  area  and  are  willing 
to  develop  a  MENS.  Validation  of  the  analysis  procedures  is 
not  dependent  on  any  strong  underlying  theoretical  assumptions. 
However,  it  is  assumed  that  the  various  sets  of  problem 
elements  (i.e.,  alterables,  constraints,  and  needs)  can  be 
identified  and  the  interactions  amount  them  determined. 

Locations  of  analysis  effort 

The  vast  majority  of  the  work  required  to  develop  the 
MENS  using  the  Unified  Program  Planning  approach  (Hill  and 
Warfield,  1972)  could  be  accomplished  at  or  below  the  service- 
headquarters  level.  Some  services  have  requirements  commands 
to  carry  out  much  of  the  analysis  effort.  Inputs  to  the 
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threat  assessment  must  come  from  the  various  intelligence 
agencies,  such  as,  the  Central  Intelligence  Agency  (CIA), 

Defense  Intelligence  Agency  (DIA) ,  and  the  National  Security 
Agency  (NSA) .  The  inputs  for  existing  capabilities  could 
come  from  the  operational  and  logistical  commands  of  the 
services . 

The  role  of  the  decision  makers 

The  top-level  managers  in  OSD  or  the  service  head¬ 
quarters  would  have  very  little  involvement  in  the  initial 
analyses  of  this  task.  However,  as  preparation  for  the  MENS 
advanced,  top-level  DOD  managers  should  become  more  actively 
involved.  This  involvement  in  the  analysis  effort  would 
provide  the  decision  makers  with  the  necessary  background  for 
an  evaluation  of  the  MENS  as  described  in  the  next  task. 

The  role  of  the  systems-engineering 
teams 

The  systems-engineering  teams  to  be  used  at  the  service 
headquarters  would  be  responsible  for  developing  the  procedure 
for  implementing  the  Unified  Program  Planning  approach  to  the 
development  of  the  MENS.  They  would  also  be  responsible  for 
providing  technical  support  to  the  various  groups  involved  in 
developing  the  MENS.  Finally,  they  would  review  each  MENS 
prior  to  its  submission  to  the  SERVSEC  and  SECDEF.  This 
review  should  be  designed  to  insure  consistency  in  technical 
approach,  format,  content,  and  level  of  detail.  This  team 
would  then  assist  the  decision  makers  in  understanding  the 
analysis  procedures  required  to  develop  the  MENS. 


Cost  and  time  considerations 

The  implementation  of  the  Unified  Program  Planning 
approach  to  MENS  development  should  not  cause  significant 
increases  in  the  costs  or  time  of  the  overall  process.  The 
existing  operational-requirements  analysis  personnel,  assisted 
by  the  projected  systems-engineering  teams,  should  be  able 
to  develop  and  document  a  MENS  using  the  suggested  approach. 
Once  personnel  become  familiar  with  the  standardized  analysis 
approach  of  Unified  Program  Planning,  the  MENS-development 
time  should  be  shortened.  The  existing  question  as  to  what 
constitutes  a  MENS  would  be  alleviated  by  this  approach. 

Policy  implications 

The  recommendations  embodied  in  the  projected  analysis 
procedure  relate  only  to  the  development  procedures  and  format 
for  the  MENS.  There  is  no  intent  here  to  revise  the  MENS 
processing  and  approval  cycle.  Therefore,  the  policy  implica¬ 
tions  are  minimal  and  relate  only  to  the  requirement  to  revise 
DOD  Directives  5000.1  and  5000.2  to  reflect  the  Unified 
Program  Planning  approach  to  developing  the  MENS. 

References  to  the  literature  for 
the  analysis  procedure 

The  Unified  Program  Planning  approach  was  originally 
documented  by  Hill  and  Warfield  in  1972.  Warfield  placed  this 
approach  in  a  larger  context  in  his  Societal  Systems  text  in 
1976.  A  recent  article  by  Hill  and  Ollila  (1978)  addressed 
the  topic  of  complex  decision-making  processes  by  extending 
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the  interaction-matrices  approach.  The  Nominal  Group  Process 
outlined  above,  as  a  means  to  develop  inputs  for  the  Unified 
Program  Planning  procedure,  has  been  discussed  by  Delbecq, 

Van  de  Van,  and  Gustafson  in  their  text.  Group  Techniques  for 
Program  Planning  (1976). 

Task  #2:  To  Develop  an  Analysis  Procedure 
to  Trade  Off  Newly  Identified  Mission- 
Element  Needs  Early  in  the  Sy  stems  - 
Aquisition  Process 

Introduction 

This  task  addresses  the  development  of  an  analysis 
procedure  to  provide  means  of  evaluation  at  the  major-decision 
Milestone  0.  This  evaluation  would  enable  the  tradeoff  of 
newly  identified  mission-element  needs  and  determine  if  a 
program  should  proceed  to  the  conceptual  phase.  The  task  is 
addressed  in  the  DSARC  objective  presented  in  Figure  18  (p.137) 
that  states 

to  insure  that  the  minimum  set  of  systems-acquisition 
programs  necessary  to  meet  National  Defense  requirements 
are  approved  and  the  low  priority,  least  affordable  systems 
are  cancelled. 

The  recommended  approach  to  this  evaluation  process 
has  several  steps.  First,  semiannual  MENS  submission  dates 
would  be  established;  as  a  draft  MENS  (potential  programs) 
is  submitted  by  a  service,  it  would  be  assigned  a  number 
specifying  the  particular  year  and  group.  At  the  next 
six-months-review  period,  it  would  be  considered.  Second,  on 
a  semiannual  basis,  OSD  would  review  and  evaluate  all  the 
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MENS  (potential  programs)  using  the  standardized  evaluation 
criteria  and  procedures  developed  below.  Finally,  programs 
showing  sufficient  merit  would  be  advanced  to  the  conceptual 
phase . 

This  approach  would  be  particularly  beneficial  during 
the  next  several  years  as  all  potential  programs  go  through 
the  MENS  process  for  the  first  time.  The  approach  would 
provide  important  features  to  the  systems-acquisition  process. 
It  would  enable  at  least  an  attempt  to  be  made  toward 
trading  off  the  candidate  programs  and  determining  their  rela¬ 
tive  priority.  This  approach  should  secure  the  objective  of 
developing  only  the  required  minimum  set  of  systems-acquisition 
programs  and  cancelling  the  low-priority  programs.  Finally, 
each  approved  systems-acquisition  program  would  be  assigned 
specific  year  groups,  which  would  allow  it  to  be  more  effec¬ 
tively  integrated  into  the  Congressional  budgetary  cycle 
and  the  PPBS . 

Relationship  to  the  generalized 
systems-engineering  steps 

Considering  the  systems-engineering  framework  analysis 
in  Chapter  3  (p.62),  the  procedure  outlined  in  this  task 
addresses  two  of  the  general  steps  of  the  systems-engineering 
logic  dimension.  The  impact-assessment  step  is  addressed 
by  the  structuring  of  the  problem,  criteria  identification, 
and  program-evaluation  efforts  associated  with  the  task.  The 
output-specification  step  is  addressed  by  the  decision  process 


used  to  select  the  programs  to  be  advanced  to  the  conceptual 
phase . 

Analysis-procedure  selection 
considerations 

Based  on  the  program  characteristics,  two  important 
considerations  are  data  availability  and  the  stage  of  the 
acquisition  process.  Data  availability  at  this  point  is  rather 
limited  and  data  are  qualitative.  Two  organizational  considera¬ 
tions  are  important  in  selecting  an  analysis  procedure.  First, 
the  availability  of  the  analyst  must  be  considered  because 
much  of  the  structuring  of  the  analysis  problem  will, 
of  necessity,  have  to  be  done  within  OSD  where  the  decision 
to  approve  the  programs  is  to  be  made.  Second,  the  availability 
of  top-level  OSD  decision  makers  must  be  considered.  It  would 
be  inappropriate  to  select  a  procedure  that  placed  unjustified 
demands  on  the  professional  time  of  these  decision  makers. 

Analysis  procedure 

The  analysis  procedure  selected  for  application  in 
this  particular  task  is  that  of  the  Justification  (Viability/ 
Objectives)  Checklist  as  developed  by  Bradbury,  Gallagher,  and 
Suckling  (1973).  This  particular  approach  was  selected  be¬ 
cause  it  would  require  only  qualitative  information.  This 
procedure  would  therefore  support  both  the  generalized  steps 
of  impact  assessment  and  of  output  specification  and  inter¬ 
pretation  for  the  mission-analysis  (program-planning)  phase. 
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Results  and  anticipated  benefits 

This  analysis  procedure  would  result  in: 

1.  A  set  of  comprehensive  checklists  that  would 
qualitatively  describe  the  various  aspects  of  each  program 
under  consideration. 

2.  A  derived  priority  list  of  all  the  programs  under 
consideration . 

3.  An  assessment  of  the  potential  programs  that  had 
not  met  a  preselected  set  of  standards  and  were  deleted. 

This  procedure  would  increase  the  ability  to  discrimi¬ 
nate  among  the  various  potential  programs  and  more  efficiently 
evaluate  their  viability  and  overall  contribution  to  the 
national  defense. 

Capsule  description  of  the 
analysis  procedure 

Several  steps  would  be  required  to  assess  the  various 
potential  programs  by  using  the  checklist  procedure.  First, 
a  preanalysis  effort  should  be  directed  toward  identifying 
and  defining  the  evaluation  problem.  This  step  should  include 
the  needs,  constraints,  alterable  quantities,  and  objectives. 
Evaluation  areas  (Table  9)  and  associated  factors  would  then 
be  determined.  Each  factor  would  have  levels  of  goodness 
associated  with  it  dependent  on  the  evaluation  area.  A 
baseline  subset  of  factors  and  levels  of  goodness  would  be 
established  as  the  minimum  acceptable  standard  for  the  approval 
of  a  program.  The  decision  maker  would  then  assess  each  factor 
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in  relation  to  each  potential  program  and  determine  what 
level  of  goodness  could  be  achieved.  This  process  would  be 
carried  out  for  all  potential  programs  until  a  comprehensive 
checklist  of  factors  has  been  compiled.  These  completed 
checklists  would  then  provide  a  basis  for  determining  if  a 
particular  program  met  the  minimum  acceptable  standard  for 
approval  to  advance  to  the  conceptual  phase.  The  check¬ 
lists  would  also  provide  a  means  for  the  decision  makers  to 
discriminate  among  the  potential  programs  and  establish  a 
priority  listing  of  programs  for  approval. 

Input  requirements 

The  preanalysis  effort  mentioned  above  for  this  task 
would  be  effectively  accomplished  by  developing  the  MENS  in 
accordance  with  the  provisions  of  the  first  task.  The  Unified 
Program  Planning  approach  would  provide  the  needs,  constraints, 
alterables,  and  objectives  of  the  problem  along  with  their 
interrelationships.  Brainstorming,  the  Nominal  Group  Process, 
or  other  means  could  be  used  to  identify  the  evaluation  areas 
and  their  associated  factors.  The  same  techniques  used  for 
generating  ideas  could  also  be  used  to  develop  the  various 
levels  of  goodness  associated  with  each  factor. 

Table  9  presents  a  sample  list  of  factors  that  could 
be  used  to  evaluate  potential  systems-acquisition  programs. 
However,  this  list  does  not  present  all  appropriate  factors 
but  is  simply  a  representative  list  of  the  evaluation  factors 
that  should  be  considered.  Table  9  shows  only  two  levels  of 


goodness  associated  with  each  factor.  Under  some  conditions, 
the  scale  could  be  broadened  to  include  several  levels  of 
goodness.  As  an  example,  Bradbury,  Gallagher,  and  Suckling 
(1973)  indicated  the  following  six  possible  scale  levels  in 
relation  to  viability:  no  problem,  minor  and  major  problems 
(in  relation  to  performance) ,  minor  and  major  threats  (in 
relation  to  the  availability  of  solutions) ,  and  ignorance 
(as  a  measure  of  uncertainty) . 

Operational  and  analysis  assumptions 

From  the  operational  standpoint  of  DOD,  it  is  assumed 

that  there  is  a  set  of  potential  programs  that  must  be 

evaluated.  Normally  these  potential  programs  would  be 

presented  in  a  MENS.  The  analysis  procedures  are  not  dependent 

on  any  strong  underlying  theortical  assumptions  for  validity 

of  employment.  However,  it  is  assumed  that  meaningful  and 
2 

standardized  evaluation  areas  and  factors  may  be  identified. 
Finally,  it  is  assumed  that  two  or  more  levels  of  goodness 
can  be  associated  with  each  factor  so  as  to  scale  that  factor. 

Location  of  the  analysis  effort 

The  actual  analysis  effort  to  support  this  task  could 
be  carried  out  jointly  by  the  services  and  OSD.  The  services 
could  suggest  evaluation  areas  and  factors;  however,  the 
final  structuring  of  the  checklists  should  be  done  within  OSD. 

2 

Standardized  in  this  case  refers  to  the  requirement  that  the 
factors  identified  should  be  equally  applicable  to  all  programs  subjected 
to  evaluation. 
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Role  of  the  decision  makers 

The  top-level  managers  in  the  OSD  would,  of  necessity, 
be  involved  in  developing  and  filling  out  the  checklists  and 
would  make  recommendations  to  the  SECDEF  on  a  potential  program. 
This  could  be  done  by  a  simple  change  in  operational  procedures 
without  a  revision  of  existing  policies.  The  SECDEF  would 
then  make  a  final  determination  on  the  approval  of  a  specific 
program  based  on  its  MENS  and  the  recommendations  derived  from 
the  checklists. 

The  role  of  the  systems-engineering 
teams 

The  systems-engineering  team  to  be  used  within  OSD 
would  be  deeply  involved  in  the  initial  phases  of  this  analysis 
task.  It  would  be  responsible  for  coordinating  the  overall 
development  of  the  checklist,  including  the  consistency, 
reliability,  and  validity  of  the  various  evaluation  factors. 

The  team  would  insure  that  the  decision  makers  were  properly 
educated  on  the  purpose,  intent,  and  employment  of  the  check¬ 
lists.  Once  the  checklists  were  fully  developed,  the  team 
would  assist  the  decision  makers  in  applying  them  to  the 
evaluation  process  for  the  potential  programs. 

Cost  and  time  considerations 

This  procedure/  which  would  require  a  good  deal  of  an 
analyst's  time  and  dedicated  support,  is  addressed  in  greater 
detail  in  the  next  section.  Initially  it  would  also  require 
some  additional  time  from  the  decision  makers  until  the 


2] 


standardized  checklists  were  fully  developed.  However, 
once  the  evaluation  process  has  been  fully  implemented, 
the  decision  makers  would  not  be  contributing  an  excessive 
amount  of  time  to  this  decision  function.  The  checklists 
should  complement  and  amplify  the  MENS,  thereby  providing 
a  more  concise  and  well-defined  decision  package  for  the 
top-level  OSD  managers  to  consider. 

Policy  implications 

The  only  significant  policy  implication  associated 
with  this  task  is  the  requirement  to  establish  semiannual 
due  dates  for  the  MENS.  This  requirement  could  be  ful¬ 
filled  by  a  revision  to  DOD  directives  5000.1  and  5000.2. 

References  to  the  literature 
for  the  analysis  procedure 

The  checklist  approach  suggested  here  is  adapted 
from  the  procedure  developed  by  Bradbury,  Gallagher,  and 
Suckling  (1973).  Allen  and  November  (1969),  Mickler  (1972) 
Varble  (1972) ,  Augood  (1973)  and  Clarke  (1974)  also  wrote 
about  evaluation  factors  and  the  development  of  checklists 
for  the  evaluation  and  selection  of  R&D  programs. 
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Task  # 3 :  To  Devleop  an  Analysis  Procedure 
to  Support  the  DSARC / ( S ) SARC  Decision 
Process  at  Major-Decision 
Milestones  I  and  II 


Introduction 

This  task  addresses  the  development  of  an  analysis 
procedure  that  will  support  the  DSARC/ (S) SARC  decision  process 
at  major-decision  Milestones  I  and  II.  Although  there  are 
two  distinct  decision  points  to  consider,  they  are  quite 
similar.  They  both  occur  during  that  portion  of  the  overall 
systems-acquisition  process  when  there  is  a  good  deal  of  risk 
and  uncertainty  with  the  program.  This  is  also  the  period  in 
the  acquisition  process  when  there  is  a  transition  from  the 
availability  of  qualitative  to  quantitative  data. 

Another  similarity  between  major  Milestones  I  and  II 
is  that  the  same  types  of  decisions  are  required  to  be  made 
at  both  points.  The  first  decision  deals  with  the 
evaluation  and  selection  of  alternative  candidate  solutions 
for  advancement  to  the  next  phase  of  the  acquisition  process. 
The  second  decision  deals  with  the  go/no-go  determination 
for  the  overall  program  to  proceed  to  the  next  phase. 

Because  of  the  similarities  between  major-decision 


Milestones  I  and  II,  the  same  type  of  analysis-support 
procedures  will  be  used  for  both. 
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Relationship  to  the  generalized 
systems-engineering  steps 

Considering  the  systems-engineering-f ramework 
analysis  of  Chapter  3  (p.62),  the  procedure  outlined  in  this 
task  addresses  two  of  the  general  steps  of  the  systems- 
engineering  logic  dimension.  The  impact-assessment  step 
is  addressed  by  the  structuring  of  the  problem,  criteria 
identification,  and  program-evaluation  efforts  associated 
with  this  task.  The  output-specification  step  is  addressed 
by  the  decision  process  for  selecting  the  alternative  candidate 
solutions  to  proceed  to  the  next  phase  and  making  the  go/no-go 
determination  for  the  overall  program  to  advance  to  the 
following  phase. 

Analysis-procedure- selection  considerations 

Based  on  the  program  characteristics,  there  are 
three  important  considerations:  availability  of  data,  stage 
of  the  acquisition  process,  and  the  extent  of  the  resources  to 
be  committed  to  the  program.  More  data  are  available  at 
these  decision  points  than  at  the  Milestone  0  decision  point, 
but,  because  of  the  stage  of  the  acquisition  process,  much 
data  are  still  qualitative.  The  commitment  of  significant 
resources  is  required  for  a  program  to  advance  to  the  subse¬ 
quent  phase.  Consequently  the  analysis  procedures  selected 
to  support  these  decision  points  must  have  sufficient  depth 
to  justify  the  commitment  of  these  significant  resources.  The 
following  organizational  considerations  are  important:  how 
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critical  is  the  decision  to  the  organization,  how  available 
is  the  decision  maker ' s  time,  and  how  available  is  the  analysis 
support.  Basically,  the  significance  of  the  decision  should 
be  correlated  to  the  decision  maker's  time  and  the  analysis 
support.  Because  the  decisions  of  Milestones  I  and  II 
lead  to  the  approval  of  a  system  for  full-scale  development, 
an  exacting  analysis  procedure  is  justified  to  support 
the  decision-making  process  at  these  two  milestones. 

Analysis  procedure 

The  analysis  procedure  for  application  in  this  parti¬ 
cular  task  is  that  of  project-scoring  models  as  addressed  by 
Clarke  (1974)  and  Moore  and  Baker  (1969).  Scoring  models 
require  decision  makers  to  relate  each  project's  merit  to 
several  project  characteristics  (factors  or  criteria) . 

Criterion  scores  can  then  be  aggregated  to  yield  an  overall 
project  score.  The  choice  of  this  approach,  with  its 
increased  complexity  and  greater  requirement  for  analysis 
effort,  reflects  the  greater  importance  of  these  decisions  to 
the  organizations  and  the  stage  of  the  acquisition  process. 

It  also  makes  the  best  use  of  the  partially  qualitative  nature 
of  the  available  data. 

Results  and  anticipated  benefits 

This  analysis  procedure  should  produce: 

1.  A  preference  for  which  of  the  alternative  candidate 


solutions  should  be  advanced  to  the  next  phase. 
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2.  A  determination  of  whether  or  not  the  overall 
program  should  be  advanced  to  the  next  state  (phase) . 

This  procedure  would  enhance  the  ability  to  discriminate 
among  the  various  alternative  candidate  solutions  and  between 
the  merits  of  approving  or  not  approving  the  overall  program 
for  advancement  to  the  next  phase. 

Capsule  description  of  the 
analysis  procedure 

Several  phases  are  required  to  develop  the  project 
scores.  First,  the  Unified  Program  Planning  (p.192)  could  be 
used  in  a  preanalysis  phase  to  identify  and  define  the  decision 
situation.  Next,  an  optional  preparatory  phase  would  be 
required  in  which  the  project  score  forms  would  be  developed, 
if  necessary.  Table  9  shows  several  suggested  evaluation  areas 
and  associated  factors  to  be  identified  although  it  is  not 
a  comprehensive  listing.  As  many  as  five  to  seven  scale 
values  could  then  be  developed  for  each  factor. 

The  next  phase  would  determine  the  factor  weights 
within  each  evaluation  area.  Here,  each  individual  decision 
maker  would  rank  the  factors  in  each  evaluation  area;  each 
factor  could  then  be  converted  to  a  numerical  value  assuming 
equal  intervals  between  adjacent  ranks.  Group  evaluations 
could  be  calculated  by  averaging  the  set  of  numerical  weights, 
assuming  that  each  decision  maker  had  an  equal  degree  of 
knowledge.  The  list  of  factors  with  their  associated 
weights  and  scaling  values  would  make  up  the  overall  project 
score  sheet  (Table  9)  . 
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The  final  phase  of  the  analysis  procedure  would  be 
to  evaluate  each  project  or  alternative  at  each  decision  point. 
First,  the  decision  makers  should  concur  on  the  value  assigned 
to  each  evaluation  factor  associated  with  each  project  or 
alternative.  A  score  could  then  be  developed  for  each 
evaluation  area  by  multiplying  the  factor  weight  by  the 
corresponding  value  and  then  adding  all  factors.  Total 
project  scores  would  be  obtained  by  summing  the  evaluation 
areas.  The  decision  makers  could  assign  a  decision  variable 
for  the  risk,  r,  that  could  be  introduced  into  the 
problem;  this  would  premultiply  each  evaluation  area  score. 
Higher  values  of  r  would  indicate  less  risk  for  that  parti¬ 
cular  evaluation  area  and  lower  values,  greater  risk. 

A  mathematical  representation  of  the  scoring  model 
is  presented  as  follows: 


S  .  =  ^  r  .  (f  w  . ,  v .  . .  ) 
1  :  3  k  :k  ijk 


where  is  the  total  score  for  the  i  project;  r^  is  the 

th 

risk  factor  for  the  j  evaluation  area  (all  values  of  r^  would 

th  th 

sum  to  one);  w.,  is  the  weight  of  the  k  factor  in  the  j 
D  K 

evaluation  area;  and  v^^  is  the  scaling  value  for  the  k 

t  h  th 

factor  in  the  j  evaluation  area  for  the  i  project. 


The  procedure  described  above  could  be  directly 
applied  to  the  types  of  decisions  used  for  evaluating  and 
selecting  alternative  candidates  to  be  advanced  to  the  next 
phase.  However,  the  evaluation  factors  would  require  some 
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revision  from  the  Milestone  I  decision  problem  to  the 
Milestone  II  decision  problem. 

A  procedure  to  support  a  decision  for  a  go/no-go 
determination  for  an  overall  program  to  advance  to  the  next 
phase  would  also  require  preanalysis.  Initially,  a  group  of 
decision  makers  must  determine  a  baseline  subset  of  factors 
and  associated  scaling  values  to  be  used  to  establish  the 
minimum  acceptable  standard  score  to  advance  the  overall 
program  to  the  next  phase.  The  factors  chosen  and  the 
standard  score  would  then  be  used  to  gauge  the  probability  of 
success  of  the  program  in  the  next  acquisition  phase.  As 
more  programs  are  subjected  to  this  analysis  procedure,  the 
factors  necessary  to  calculate  success  probability  would  become 
more  readily  identifiable.  The  minimum  standard  score 
necessary  to  forecast  a  reasonable  degree  of  future  success 
would  also  become  more  visible.  An  optimum  set  of  baseline 
factors  and  the  minimum  standard  score  to  project  the  success 
of  the  program,  within  some  reasonable  limits,  could  then  be 
selected . 

Input  requirements 

The  preanalysis  phase  mentioned  above  would  be  effec¬ 
tively  accomplished  by  using  the  Unified  Program  Planning 
approach.  With  this  approach  the  MENS  could  be  updated  to 
properly  define  the  system-development  problem  under  considera¬ 
tion.  The  evaluation  areas,  evaluation  factors,  and  scaling 
values  associated  with  each  factor  would  have  to  be  developed; 
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the  Nominal  Group  Process,  or  another  means  of  generating  ideas, 
could  be  used  to  develop  the  evaluation  factors  and  the  various 
scaling  levels.  Table  9  presents  a  representative  list  of 
evaluation  factors  and  scaling  level-terms  applicable  to 
Defense  systems  acquisitions. 

Operational  and  analysis 
assumptions 

From  the  operational  standpoint  of  DOD,  it  is  assumed 
that  a  set  of  alternative  candidate  solutions  associated  with 
a  specific  acquisition  program  requires  evaluation  to 
determine  that  should  be  advanced  to  the  next  acquisition 
phase.  It  is  also  assumed  that  a  go/no-go  determination 
must  be  made  on  whether  the  overall  acquisition  program 
should  proceed  to  the  next  phase. 

The  analysis  procedures  to  develop  the  project 
scores  assume  a  linear  model.  Therefore,  the  projects  are 
ranked  on  an  ordinal  scale  and  there  is  no  way  to  determine 
how  much  better  one  project  is  than  another.  In  identifying 
and  using  the  evaluation  factors,  it  is  assumed  that  they  are 
mutually  exclusive  and  score  independently. 

Since  the  project  score  is  a  single  performance 
measure,  there  is  no  way  to  gain  a  utility  measure  of  the 
various  projects  using  the  scoring  model  (Dean  and  Nishry, 

1965) .  Finally,  the  scoring  model  will  not  effectively 
reflect  the  risk  associated  with  these  stages  of  the 
acquisition  process  except  through  the  introduction  of  a 


22  5 


decision  variable  for  the  risk  factor,  r,  as  discussed 
above . 

Location  of  the  analysis  effort 

The  actual  evaluation  of  the  alternative  candidate 
solutions  would  begin  in  the  systems-program  offices. 

These  program  offices,  and  other  service  elements,  could 
assist  in  identifying  and  developing  the  evaluation  areas  and 
factors.  The  systems-program  offices  could  use  the  analysis 
procedure  noted  above  to  arrive  at  their  initial  recommenda¬ 
tion  on  which  alternative  candidate  solutions  are  to  enter 
the  next  phase.  However,  to  support  the  DSARC  and  (S)SARCs 
decision  processes,  much  of  the  analysis  for  this  task  would 
have  to  be  carried  out  within  OSD  and  at  the  several  service 
headquarters,  respectively.  These  councils  should  at  least 
validate  the  recommendation  and  make  their  own  decisions  on 
the  go/no-go  determination  for  the  overall  program.  Therefore, 
the  structuring  of  the  decision  problem,  the  final  develop¬ 
ment  of  the  project  scoring  sheets,  and  the  actual  use  of  the 
scoring  models  would  have  to  be  accomplished  within  OSD  and 
the  several  headquarters  elements. 

The  role  of  the  decision  makers 

As  indicated  above  in  the  description  of  the  analysis 
procedure,  the  members  of  the  DSARC  and  the  (S)SARCs  would  be 
involved  in  several  phases  of  the  analysis  effort.  Initially, 
they  would  need  to  be  educated  in  the  overall  procedure;  then 


they  should  be  involved  in  identifying  and  developing  the 
evaluation  areas  and  factors.  Finally,  they  would  have  to 
evaluate  the  alternative  candidate  solutions  and  the  overall 
programs . 

The  role  of  the  systems- 
engineering  teams 

The  systems-engineering  teams  recommended  for  OSD  and 
the  several  service  headquarters  would  be  deeply  involved  in 
supporting  this  procedure  throughout  its  use.  Initially, 
the  team  members  would  have  to  educate  their  decision  makers 
(the  DSARC/(S) SARC  members)  in  the  purpose,  intent,  and 
employment  of  the  procedure.  They  would  have  to  insure  that 
the  general  systems-engineering  step  of  input  description 
and  specification  used  the  Unified  Program  Planning  approach. 

The  team  members  should  coordinate  the  development  of  the 
evaluation  areas,  the  factors,  and  the  associated  scaling 
values  for  the  factors.  Finally,  the  analysts  should  assist 
the  decision  makers  in  using  the  scoring  models. 

Cost  and  time  considerations 

This  procedure  would  require  a  good  deal  of  an  analyst's 
time  and  associated  funding  support.  Initially,  the  decision 
makers  would  need  time  to  familiarize  themselves  with  the 
procedure  and  to  develop  the  evaluation  areas,  factors,  and 
scaling  values  associated  with  the  factors.  However,  once  the 
evaluation  process  is  fully  implemented,  the  decision  makers 
would  only  need  to  evaluate  and  select  the  alternative 
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candidate  solutions  and  make  the  go/no-go  determinations  for 
the  programs.  This  function  could  be  accomplished  in  approxi¬ 
mately  the  same  time  that  is  now  devoted  to  the  evaluation 
and  selection  process. 

Policy  implications 

Mo  policy  revisions  would  be  necessary  to  implement 
this  analysis  procedure.  Only  certain  procedures  in  the 
decision  process  would  need  revision. 

References  to  the  literature 
for  the  analysis  procedure 

The  capsule  description  of  the  analysis  procedure 
is  partially  adapted  from  Dean  and  Nishry  (1965).  Baker  and 
Pound  (1964),  Pound  (1964),  and  Moore  and  Baker  (1969) 
all  addressed  the  development  and  employment  of  scoring 
models.  Augood  (1973),  Baker  (1974),  Clarke  (1974),  and 
Baker  and  Freeland  (1975)  all  assessed  the  use  of  scoring 
models  for  evaluating  and  selecting  R&D  projects.  These 
latter  authors  also  presented  scoring  models  in  the  context 
of  the  overall  set  of  project  evaluation  and  selection  tech¬ 
niques  . 

Task  #4:  To  Develop  an  Analysis  Procedure  to 
Support  the  DSARC / ( S ) SARC  Decision  Process 
at  Maj  or-Deaision  Milestone  III 

Introduction 

This  task  develops  an  analysis  procedure  to  support 
DSARC/ (S) SARC  decision  process  at  major-decision  Milestone  III. 
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Only  one  type  of  decision  needs  to  be  made  at  this  milestone — 
the  go/no-go  determination  for  the  overall  program  to  advance 
to  the  next  phase. 

The  decision  at  this  milestone  is  whether  to  produce 
and  deploy  the  system.  During  the  previous  acquisition  phase, 
the  service  responsible  should  have  fully  developed  the 
single  alternative  candidate  solution  that  would  now  be 
considered  for  production.  Risk  should,  therefore,  be  at 
its  lowest  point  and  the  number  of  evaluation  factors  should 
be  reduced  as  there  are  no  alternative  candidates  involved. 
However,  the  commitment  of  resources  will  be  high  for  both 
the  production  and  operational  deployment  phases. 

Relationship  to  the  generalized 
systems-engineering  steps 

Considering  the  systems-engineering-f ramework  analysis 
of  Chapter  3,  the  procedure  outlined  in  this  task  addresses 
the  generalized  step  of  output  specification  and  interpreta¬ 
tion.  This  is  because  the  decision  under  consideration  is  a 
go/no-go  determination.  However,  the  previous  steps  in  the 
systems-engineering  logic  dimension  must  be  carried  out  to 
provide  a  foundation  and  framework  for  this  particular 
decision. 

Analysis-procedure  selection 
considerations 

Based  on  the  program  characteristics,  important 
considerations  are  the  stage  of  the  program  and  the  extent 
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of  the  resources  to  be  committed  to  the  program.  At  this 
milestone,  a  great  deal  of  quantitative  data  should  be 
available  to  support  analysis  and  the  resource  commitment  is 
large  and  should  be  complemented  with  a  thorough  analysis. 

Considering  this  major-decision  milestone  from  an 
organizational  standpoint,  this  decision  is  critical  to  the 
service  involved  and  to  the  DOD.  It  decides  some  subsets  of 
the  next  generation  of  Defense  systems.  Therefore,  it  must 
be  as  carefully  analyzed  as  possible  in  accordance  with  the 
availability  of  decision  makers'  and  analysts'  time. 

Analysis  procedure 

Decision  analysis,  as  outlined  by  Sage,  Warfield, 
Thissen,  et  al.  (1978),  has  been  selected  for  this  particular 
task  specifically,  the  use  of  multiattribute  utility  theory. 
This  approach  will  insure  the  greatest  value  from  using 
the  evaluation  factors  presented  in  Table  9.  The  critical 
decision  situation  necessitates  the  rigorous  analysis  pro¬ 
cedure  of  the  multiattribute  utility  theory.  This  procedure 
will  require  a  good  deal  of  time  from  both  the  decision 
makers  and  the  analysts,  which  is  justifiable  when  the 
magnitude  of  the  commitment  of  resources  needed  for  pro¬ 
duction  and  the  importance  of  the  decision  to  deploy  the 
system  are  considered. 
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Results  and  anticipated  benefits 

This  analysis  should  produce: 

1.  A  model,  usually  a  decision  tree,  to  represent 
the  actual  situation  at  major-decision  Milestone  III. 

2.  An  optimal  course  of  action,  so  that  the  decision 
makers  (the  DSARC/ (S) SARC  members)  know  whether  or  not  to 
produce  the  system  based  on  their  own  utility  functions. 

3.  An  analysis  of  the  sensitivity  of  the  best  course 
of  action  to  errors  in  the  data. 

This  procedure  would  enhance  the  ability  of  the  deci¬ 
sion  makers  to  discriminate  between  the  decision  alternatives 
of  going  or  not  going  to  the  production  and  deployment  of  the 
system.  This  would  also  enable  the  decision  makers  to  discern 
the  probability  of  success  of  the  system  during  production 
and  deployment. 


Capsule  description  of  the 
analysis  procedure 

The  description  below  has  been  abstracted  from  the 
"Research  Concerning  a  User's  Guide  to  Public  Systems  Methodology", 
by  Sage,  Warfield,  Thissen,  et.  al.  (1978). 

Several  steps  or  phases  are  involved  in  conducting  a 
decision  analysis  and  are  proceeded  through  in  an  iterative 
manner.  A  pre-analysis  effort  is  directed  toward  identifying 
and  defining  the  decision  situation.  Activities  include  identi¬ 
fying  needs,  pertinent  constraints,  alterable  quantities,  and 
objectives.  The  input  specification  portion  of  a  system  effort 
yields  this  information.  A  structuring  effort  is  directed 
toward  identifying  decisions  and  outcomes  and  the  relationships 
between  them,  and  structuring  preferences  to  facilitate  evalua¬ 
tion  of  outcomes.  This  is  accomplished  in  the  impact  assess¬ 
ment  step.  Encoding  of  information  pertaining  to  the  uncertainty 
of  outcomes  and  incorporating  decision  maker  attitude  toward 


231 


risk  into  a  function  describing  the  decision  maker '  s  preferences 
are  the  principal  tasks  in  a  probabilistic  effort.  In  an 
informational  effort ,  the  impact  of  additional  information  to 
resolve  uncertainty  on  the  outcomes  or  by  a  reiteration  through 
some  or  all  steps  is  assessed.  An  optimal  strategy  is  then 
calculated  for  the  decision  maker  to  follow,  and  subsequently, 
the  decision  is  made. 

This  procedure  may  be  extended  to  incorporate  the 
input  of  several  decision  makers  and  a  variety  of  different 
evaluation  factors  or  attributes.  In  this  context,  attribute 
may  be  defined  as  the  characteristic  or  quality  that  will 
indicate  the  degree  to  which  a  systems-acquisition  program 
under  consideration  meets  the  development  objective  with 
which  the  attribute  is  associated.  When  there  are  several 
decision  makers,  as  is  the  case  in  the  DS ARC/ ( S ) SARC  decision 
process,  and  there  are  a  number  of  attributes,  then  the  deci¬ 
sion  becomes  a  group  decision  and  multiattribute  utility 
theory  is  applicable.  This  analysis  procedure  responds  to 
the  complex  and  uncertain  nature  of  the  decision  situation 
at  Milestone  III. 

Input  requirements 

Implementation  of  the  decision-analysis  procedure 
would  require  several  inputs.  First,  the  decision  makers 
and  the  analysts  would  jointly  have  to: 

1.  Define  the  decision  situation  by  using  the  Unified 
Program  Planning.  The  needs,  constraints,  objectives  (and 
their  structure),  and  the  measures  for  evaluating  their  achieve¬ 
ment  should  be  included. 
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2.  List  the  alternatives  available  to  the  decision 
maker  from  which  one,  and  only  one,  must  be  selected  at  the 
completion  of  the  analysis.  In  this  case,  the  alternatives 
are  simply  the  go  or  no-go  determinations  to  produce  and 
deploy . 

3.  List  the  possible  outcomes  of  which  one,  and 
only  one,  is  possible  after  a  decision  is  made.  In  the  case 
of  the  systems-acquisition  process,  these  outcomes  are  the 
success  or  failure  of  the  system  during  production  and 
deployment . 

4.  List  the  attributes  associated  with  the  development 
objectives  for  the  decision  situation.  These  attributes 
would  be  analogous  to  the  evaluation  factors  presented  in 
Table  9 . 

The  decision  makers,  with  the  assistance  and  guidance  of  the 
analysts,  should  also: 

1.  Answer  questions  about  the  relationship  between 
the  alternatives  and  the  outcomes. 

2.  Answer  questions  about  the  likelihood  of  the 
occurrence  of  each  outcome  based  on  the  information  available. 

3.  Answer  questions  about  the  value  of  the  outcomes. 

4.  Answer  questions  about  preferences  for  outcomes 
in  uncertain  situations  so  that  attitudes  concerning  risk 
can  be  identified. 
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Operational  and  analysis 
assumptions 

From  the  operational  standpoint  of  DOD,  it  is  assumed 
that  development  programs  moving  through  the  acquisition 
process  must  be  evaluated  by  a  group  of  decision  makers 
(DSARC/ (S) SARC  members)  to  ascertain  whether  or  not  they 
should  proceed  to  production.  The  assumptions,  on  which  this 
analysis  procedure  is  based,  were  developed  by  Sage,  Warfield, 
and  Thissen,  et  al.  (1978): 

It  is  assumed  that  the  decision  maker's  alternatives  and  the 
possible  outcomes  of  those  alternatives  can  be  identified.  It 
is  also  assumed  that  the  decision  maker  (through  personal  know¬ 
ledge,  that  of  experts,  and/or  stakeholders)  has  some  knowledge 
about  the  relationships  between  alternatives  and  outcomes,  and 
that  the  reality  of  the  decision  situation  can  be  captured  in 
a  model.  In  quantifying  decision  maker  preferences,  the  posi¬ 
tion  is  rejected  that  the  decision  maker's  preferences  can  only 
be  revealed  through  actions  already  taken.  That  is,  decision 
analysts  operate  under  the  assumption  that  the  decision  maker 
has  the  ability  to  express  preferences  for  outcomes. 


Location  of  the  analysis  effort 

The  actual  evaluation  of  the  systems-acquisition 
programs  should  begin  in  the  systems-program  offices.  These 
offices  could  do  much  of  the  preanalysis.  However,  the  bulk 
of  the  analysis  would  have  to  be  carried  out  within  OSD  and 
at  the  service  headquarters  because  the  decision  makers 
(the  DSARC/ (S) SARC  members)  would  have  to  work  closely  with 
the  analysts  (the  members  of  the  systems-engineering  teams) 
to  carry  out  the  decision-analysis  procedure. 
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The  role  of  the  decision  makers 

As  indicated  above  in  the  description  of  the  analysis 
procedure,  the  members  of  the  DSARC  and  the  (S)SARCs  would 
be  deeply  involved  in  several  phases  of  the  analysis  effort. 

They  would  have  to  work  closely  with  the  analysts  to  structure 
the  decision  problem  and  to  determine  the  decision  alternatives 
for  and  outcomes  of  the  problem.  They  v/ould  also  have  to 
supply  the  analysts  with  a  variety  of  answers  to  questions 
about  the  relationship  of  alternatives  to  outcomes,  the 
probability  of  the  occurrence  of  outcomes ,  and  the  value  to 
the  decision  makers  of  the  various  outcomes.  The  DSAJRC/ (S)  SARC 
members  should  have  sufficient  educational  background,  mana¬ 
gerial  experience,  and  R&D  knowledge  to  contribute  effectively 
to  the  decision-analysis  procedures. 

The  role  of  the  systems- 
engineering  teams 

The  systems-engineering  teams  projected  for  OSD  and 
the  several  service  headquarters  would  support  this  procedure 
throughout  its  implementation  and  utilization.  Initially, 
the  team  members  would  have  to  educate  the  decision  makers 
(the  SDARC/(S) SARC  members)  in  the  purpose,  intent,  and 
employment  of  the  procedure.  They  would  also  have  to  insure 
that  the  systems-engineering  logic  steps  are  properly  carried 
out  before  the  decision  step.  Finally,  as  indicated  previously, 
the  team  members  (analysts)  would  have  to  work  closely  with 
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the  decision  makers  in  structuring  the  decision  problem 
and  analyzing  the  optimal  course  of  action. 

Cost  and  time  considerations 

The  initial  implementation  of  this  analysis  pro¬ 
cedure  would  require  time  from  both  analysts  and  decision 
makers  because  the  decision  situation  would  have  to  be 
structured  in  detail  so  that  both  analysts  and  decision 
makers  would  clearly  understand  the  problem.  Much  of  this 
problem  structuring  would  not  have  to  be  repeated  each  time 
a  program  was  evaluated.  However,  the  use  of  this  analysis 
procedure  would  still  require  more  time  from  decision  makers 
than  is  currently  being  devoted  to  these  types  of  decisions. 

The  improvement  in  the  decision-making  capability  should 
compensate  for  the  additional  time  required  of  the  decision 
makers . 

Policy  implications 

No  policy  revisions  would  be  necessary  to  implement 
this  analysis  procedure.  The  implementation  would  require 
only  that  certain  procedures  in  a  decision  process  be  revised. 

References  to  the  literature  for 
the  analysis  procedure 

Sage,  Warfield,  Thissen,  et  al .  (1978)  presented  an  over¬ 

view  of  this  analysis  procedure.  Raiffa  (1968)  wrote  a  basic 
text  for  decision  analysis;  multiattribute  utility  theory  is 
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addressed  in  depth  by  Keeney  and  Raiffa  (1976);  and  Sage 
(1977)  covered  a  variety  of  topics  related  to  decision  analysis. 
The  IEEE  Transactions  on  Systems ,  Man  and  Cybernetics  presented 
a  "Special  Issue  on  Behavioral  Decisionmaking"  (1977). 
Litchfield,  Hansen,  and  Beck  (1976)  suggested  using  a  decision- 
analysis  model  in  the  R&D  environment  and  Keefer  (1978)  also 
addressed  the  use  of  a  decision-analysis  model  to  allocation 
planning  for  R&D  with  uncertainty  and  multiple  objectives. 

Roesch  (1977)  applied  decision  analysis  to  a  project  evaluation- 
and-selection  problem  of  the  Defense  systems-acquisition  process 

In  the  context  of  the  DOD  environment,  much  of  the 
analysis  effort  described  above  would  be  accomplished  prior 
to  and  outside  of  the  formal  DSARC/ ( S) SARC  proceedings.  How¬ 
ever,  the  results  of  the  analysis  efforts  would  be  used 
during  the  formal  DSARC/ (S ) SARC  proceedings  to  support  the 
formal  decision  process.  The  decisions,  which  resulted  from 
the  proceedings,  would  be  documented  in  the  DCP .  This  decision 
vehicle  would  then  be  processed  in  accordance  with  the 
procedures  previously  discussed.  Some  portion  of  the  analysis 
supporting  each  decision  would  be  included  in  the  DCP  when 
it  was  forwarded  to  higher  authority. 

The  several  analysis  tasks  outlined  above  are  not 
a  comprehensive  set  of  tasks  that  could  be  applied  to 
support  the  systems-acquisition  decision  process.  Rather, 

-.hese  tasks  are  representative  of  the  types  of  analysis 

■  ••■'iires  that  could  be  applied  to  support  the  decision  process 


237 


Implementation  of  the  Methodology 

The  analysis  procedures  which  constitute  the  systems- 
engineering  methodology  for  the  Defense  systems-acquisition 
process  are  outlined  in  the  previous  section.  These  analysis 
procedures,  or  others  that  could  be  developed  at  a  future 
date,  are  of  little  value  unless  they  are  effectively  imple¬ 
mented.  A  number  of  papers  on  this  topic  were  included  in  the 
work  edited  by  Schultz  and  Sleven  (1975).  Huysmans  (1970) 
specifically  addressed  the  introduction  of  operations  research 
methods  into  organizations  to  assist  with  management-related 
decisions . 

This  section  addresses  the  general  area  of  implementa¬ 
tion,  specifically,  the  implementation  of  the  methodology 
developed  in  the  previous  section.  The  following  topics 
are  discussed:  The  consideration  of  a  major  barrier  to 
effective  implementation,  a  recommendation  for  the  formation 
of  a  systems-engineering  (multidisciplinary)  team  to  minimize 
this  barrier  and  to  implement  the  methodology,  and  the 
identification  of  general  characteristics  and  functions 
of  the  team.  The  functions  needed  for  the  systems-engineering 
team  to  implement  the  methodology  in  the  DOD  environment  are 
listed.  Finally,  the  means  for  measuring  the  effectiveness 
of  the  systems-engineering  methodology  once  it  is  implemented 
will  be  evaluated. 

Many  problems  could  inhibit  an  analysis  procedure  from 
being  effectively  used  to  assist  in  the  policy  and 


decision-making  processes.  Several  of  these  inhibiting 
factors  are  the  lack  of  quantifiable  information  available 
on  which  to  base  decisions,  the  unique  nature  of  the  organiza¬ 
tion  of  problems,  and  the  lack  of  resources  for  analysis 
efforts,  including  qualified  personnel.  However,  in  the 
author's  opinion,  the  major  barrier  to  using  analysis  proce¬ 
dures  for  establishing  policies  and  making  decisions  is  the 
separation  between  the  managers  and  the  analysts  within  an 
organization.  In  many  cases,  their  activities  and  functions 
do  not  interrelate.  They  may  not  be  motivated  toward  common 
goals,  and  the  communication  between  the  two  groups  is  poor 
at  best  (Rosenzweig,  1967) . 

An  example  can  be  drawn  from  the  section  on  The  Criteria 
and  Organizational  Considerations  for  Applying  the  Systems- 

Engineering  Tools  and  Techniques  (p.  167)  to  illustrates  the  problem. 
If  only  criteria  for  the  acquisition-program  characteristics 
had  been  selected,  then  the  environment  in  which  the  analysis 
takes  place  could  have  been  ignored.  Many  management  scientists 
and  operations  analysts  would  find  nothing  at  all  odd  about 
this.  However,  the  environment  of  the  analysis  is  the  organ¬ 
ization,  which  must  plan  for,  pay  for,  carry  out,  and  will, 
hopefully,  benefit  from  the  analysis. 

Why  then  would  analysts  view  the  analysis  situation 
from  the  narrow  contest  of  what  is  to  be  modelled  versus  the 
larger  prospective  of  the  organization  and  its  goals.  This 
question,  which  was  addressed  in  some  detail  by  Kast  and 
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Rosenzweig  (1974),  among  others,  is  closely  related  to  the 
behavioral  aspects  of  decision  making.  These  authors 
effectively  described  the  problems  associated  with  the  separa¬ 
tion  that  exists  in  some  organizations  between  policy  makers 
and  decision  makers  and  the  analysts. 

Several  suggestions  are  offered  as  to  why  the  problems 
exist.  First,  as  a  student,  an  analyst  is  taught  algorithmic 
approaches  that  are  presented  by  class  and  type.  Normally, 
the  student  is  led  to  anticipate  a  particular  type  of  problem 
and  is  not  usually  called  on  to  select  the  appropriate 
analysis  approach,  let  alone  to  visualize  it  in  an  organiza¬ 
tional  context.  Second ,  when  an  analyst  joins  industry, 
commerce,  or  government,  the  opportunity  seldom  arises  to 
look  at  the  decision  problem  from  the  manager's  standpoint. 

An  analyst  works  in  a  rather  cloistered  environment  at  low 
levels  of  the  organization  where  decisions  are  simply  regarded 
as  problems  to  be  modelled  and  then  analyzed  by  various  tech¬ 
niques.  This  prospective  is  restrictive,  causing  the  analyst 
to  omit  significant  factors  when  modelling  the  decision  situa¬ 
tion  and  leading  to  difficulties  in  communicating  plans, 
procedures,  and  results  to  decision  makers  and  policy • makers . 

On  the  other  hand,  the  decision  maker  or  policy  maker 
generally  has  a  broader,  more  varied  education  with  less 
formal  training  in  analysis  and  quantitative  methods.  Addi¬ 
tionally,  the  professional  experience  gained  in  the  upper 
echelons  of  industry,  commerce,  or  government  is  broader  than 
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that  of  the  analyst.  These  people  view  a  major  decision  as 
a  multidimensional  problem  with  possible  political,  economic, 
environmental,  and  societal  impacts.  The  decision  and  policy 
makers'  lack  of  a  strong  background  in  quantat: tive  methods 
and  their  larger  prospective  of  the  decision  problems  impede 
communication  with  the  analyst  regarding  analysis  plans, 
procedures,  and  results.  Therefore,  the  decision  makers 
and  policy  makers  fail  to  obtain  the  full  value  of  the  analysis 
efforts  within  their  organizations. 

In  summary,  an  artificial  gap  exists  between  the 
decision  makers  and  policy  makers  and  the  analysts  (manage¬ 
ment  scientists,  operations  analysts,  etc.)  within  many  organi¬ 
zations.  This  gap  needs  to  be  bridged  if  managers  are  to  gain 
the  greatest  possible  benefits  from  analyses. 

A  systems-engineering  (multidisciplinary)  team  could 
minimize  this  gap.  The  formation  of  such  a  team  is  the  first 
step  toward  bringing  the  capabilities  of  the  systems  engineer 
to  bear  on  the  problem.  Hall  (1962)  outlined  the  qualities 
that  a  systems  engineer  should  possess  as  follows : 

1.  Demonstrated  (not  merely  professed)  affinity  to  the 
systems  point  of  view. 

2.  Faculty  of  objective  judgment  and  sound  appraisal. 

3.  Imagination  and  creativity. 

4.  Facility  in  human  relations. 

5.  Effectiveness  as  a  broker  of  information. 

Gibson  (1977)  described  the  systems  team  with  subject- 
area  specialists  as  having  a  goal,  a  leader,  a  methodology, 
and  a  schedule.  Gibson  stated,  "Subject-area  knowledge  is 
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necessary,  but  not  sufficient  for  a  successful  system  study.1' 

He  pointed  out  that,  in  some  instances,  in-depth  knowledge 
is  necessary  to  study  certain  key  issues.  Several  different 
multiple  disciplines  must  be  represented  in  a  team  so  that  it 
may  implement  its  analysis  confidently  and  aggressively 
recommend  the  products  of  these  analysis  efforts  to  manage¬ 
ment.  The  multidisciplinary  aspects  of  the  team  are  also  particu¬ 
larly  important  to  the  team's  information  brokerage  role. 

The  need  for  systems-engineering  teams  in  R&D  is 
effectively  stated  by  Gibson  (1979).  These  teams  could  be 
employed  in  R&D  by  assisting  in  the  analysis  required  to 
carry  out  the  several  steps  and  phases  of  the  systems- 
engineering  methodology.  The  systems  engineer  and  his  team 
could  also  act  as  translators  for  the  analyst  and  the  decision 
maker  or  policymaker.  Shakum  (1972)  cited  the  need  for  mana¬ 
gers  at  all  levels  to  have  a  closer  relationship  with  and  a 
better  understanding  of  the  analysts  within  their  organization. 

The  systems-engineering  team  is  suggested  as  a  means 
by  which  to  directly  pursue  these  goals.  The  team  contemplated 
here  and  more  fully  described  below  has  its  most  meaningful 
role  in  large,  complex  organizations.  These  organizations 
generally  have  a  major  headquarters  where  policies  are  formu¬ 
lated  and  a  variety  of  locations  where  analysis  is  carried  out. 
The  systems-engineering  team  envisioned  here  would  normally 
be  located  at  the  headquarters. 
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Many  MS/OR  (management  science/operations  research) 
analysts,  engineers,  and  scientists  are  neither  well  equipped 
nor  motivated  to  interrelate  effectively  with  managers,  nei- 
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ther  are  managers  inclined  to  seek  out  the  analysts.  The 
members  of  a  systems-engineering  team  can  come  from  either 
group  but  should  feel  comfortable  with  both.  Figure  31  shows 
the  role  of  a  headquarters-located,  systems-engineering  team 
in  addressing  decision  problems.  This  figure  shows  multiple 
MS/OR  groups  and  other  engineering  and  scientific  analysts  at 
different  locations.  The  decision  makers  and  policy  makers 
and  the  analysts  have  their  own  and  differing  views  of  the 
decision  problems;  the  analysts  work  directly  with  the  analy¬ 
sis  models  while  the  decision  makers  and  policy  makers  view 
them  only  from  a  distance. 

The  systems-engineering  team  operates  with  both  the 
decision  makers  and  policy  makers  and  the  analysts  to  draw  out 
issues,  questions,  and  answers.  Its  primary  role  is  to  bridge 
the  gap  between  both  groups.  The  team  should  not  inhibit 
communications  between  the  two  groups  but  enhance  communica¬ 
tions  by  translating  in  both  directions  and  for  each  group. 

The  team  acts  as  an  information  broker,  as  a  mediator  of  tech¬ 
nical  matters,  as  an  interpreter  of  analysis  efforts,  and  as 
a  synthesizer  of  management  guidance.  The  need  for  this  team 
becomes  even  more  critical  when,  as  indicated  in  Figure  31, 
there  are  several  analysis  groups  at  separate  locations. 
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A  systems-engineering  team  must  have  several  distinc¬ 
tive  characteristics  if  it  is  to  bridge  the  gap  effectively. 
Each  member  of  the  team  should  possess  these  characteristics 
particularly,  the  leader  or  manager  of  the  team. 

Characteristics  of  the  Systems-Engineering  Team 

Multidiscip lined 

The  team  should  include  several  different  disciplines, 
including  MS/OR  or  computer  science,  engineering,  and  others 
as  required.  Each  member  should  be  trained  in  more  than  one 
discipline . 

Well  Educated 

Most  members  of  the  team  should  have  graduate-level 
educations  with  particular  emphasis  on  quantitative  analyses. 

A  strong  technical  background  will  allow  the  team  members  to 
be  involved  in  the  wide-ranging  technical  discussions,  brief¬ 
ings,  and  studies  necessary  to  perform  their  responsibilities. 
It  also  enables  the  systems  engineer  to  filter  out  comments 
and  information  that  are  not  pertinent  to  the  central  theme 
of  a  particular  problem  or  analysis  effort. 

Mature  and  Experienced  as  Analysts 

The  members  of  the  team  should  have  served  an  analysis 
apprenticeship  and  no w  have  moved  up  to  a  systems-engineering 
position.  Previous  experience  on  a  variety  of  programs  should 
be  a  prerequisite  for  joining  the  systems-engineering  team. 


Experienced  in  Operations  or  Management 

Members  of  the  team  should  have  previous  management 
experience  that  enables  them  to  better  grasp  the  decision 
maker  or  policy  maker's  perspective  on  the  problem.  Opera¬ 
tional  experience  in  a  field  activity  will  provide  greater 
insight  into  the  real-world  aspects  of  the  decision  problem 
under  consideration. 

Outgoing  in  Professional  Relationships 

The  members  of  the  systems-engineering  team  need  to 
actively  seek  out  contacts  and  interactions  with  both  the 
management  and  analysis  groups;  special  insight  and  under¬ 
standing  into  the  problems  of  both  groups  is  needed.  This 
type  of  in-depth  perception  can  only  be  gained  by  close  per¬ 
sonal  contacts. 

Trustworthy 

The  members  of  the  systems-engineering  team  should  be 
sincere  and  honest  in  their  relationships  with  managers  and 
analysts.  Trust  by  both  groups  will  motivate  each  group  to 
be  as  cooperative  as  possible.  So  that  the  members  of  the 
team  have  no  vested  interest  in  any  particular  program,  they 
should  be  full-time  employees  of  the  organization.  External 
consultants  should  have  an  exclusion  clause  in  their  contract 
for  all  aspects  of  the  acquisition  program,  except  those 
dealing  with  analysis. 
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Innovative 

The  systems-engineering  team  should  be  innovative  and 
creative  in  their  thinking  and  view  problems  in  a  holistic 
fashion.  They  should  have  the  ability  to  conceptualize  the 
structure  and  nature  of  problems  with  only  limited  guidance 
from  a  decision  maker,  client,  or  sponsor. 

The  size  of  the  team  is  also  important  but  it  is 
difficult  to  say  what  the  overall  size  of  a  team  at  a  major 
organizational  headquarters  should  be.  However,  when  a  speci¬ 
fic  team  is  formed  to  address  a  particular  problem  (for 
instance,  the  consideration  of  a  major  R&D  program) ,  then 
the  team  should  include  approximately  four  to  seven  profes¬ 
sional  members  with  appropriate  administrative  and  technical 
support.  If  the  team  is  any  smaller,  the  required  disciplines 
would  not  be  represented;  if  the  team  is  larger,  it  becomes 
unwieldy  and  less  productive. 

A  systems-engineering  team,  attempting  to  bridge  the 
gag  between  the  management  and  analysts  group,  should  perform 
the  functions  outlined  below: 

General  Functions  of  the  Systems-Engineering  Team 

1.  To  be  cognizant  of  ongoing  analysis  efforts  within 
the  organization.  This  requirement  is  particularly  important 
as  it  concerns  large  R&D  programs  prior  to  major-decision 
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2.  To  monitor  and  assist  in  the  implementation  of 
standardized  analysis  procedures  directed  by  organizational 
policies.  This  does  not  mean  being  responsible  for  the 
implementation  efforts  but  for  being  aware  of  how  they  are 
proceeding. 

3.  To  advise  management  on  overall  analysis  policies. 
If  the  organization  has  analysis  efforts  being  carried  out  in 
a  variety  of  locations,  analysis  policies  could  be  standard¬ 
ized. 

4.  To  keep  the  analysis  groups  informed  of  policy 
revisions  that  could  impact  analysis  procedures. 

5.  To  translate  the  results  of  internal  analysis 
efforts  into  management  information.  In  many  cases,  the 
products  of  the  analysis  groups  should  be  translated  for 
management  so  that  full  value  may  be  gained  from  the  analyses. 

6.  To  interpret  the  results  of  other  organizations' 
analyses  for  management.  Sometimes  analysis  efforts  exter¬ 
nal  to  the . organization  must  be  evaluated.  The  teams  should 
then  provide  management  with  concise  and  valid  assessments 
of  the  impact  and  meaning  of  the  external  analysis  efforts. 

7.  To  be  an  advocate  of  the  systems-engineering 
approach  and  analysis.  This  does  not  mean  to  be  a  proponent 
of  a  particular  alternative  or  algorithm  but  it  does  mean  to 
be  positive  in  discussing  the  merits  of  systems  engineering 
and  its  tools  and  techniques. 
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8.  To  educate  management  to  the  value  and  proper  use 
of  analysis  procedures. 

9.  To  carry  out  critical  analysis  procedures  that 
have  major  policy-  or  decision-making  significance  to  the  or¬ 
ganization.  On  an  infrequent  basis,  the  top-level  management 
of  the  organization  may  require  that  the  systems-engineering 
team  to  analyze  a  problem.  Typically  this  would  be  to  gain  a 
parity  check  on  an  analysis  performed  outside  the  organiza¬ 
tion  that  could  be  time  sensitive  or  highly  visible  in  its 
impact.  This  is  basically  a  misuse  of  a  team's  capabilites 
but  may  be  directed  by  top-level  management;  it  can  gain  a 
great  deal  of  good  will  within  the  organization. 

The  functions  described  above  are  general  and  appli¬ 
cable  to  a  systems-engineering  team  trying  to  bridge  the  gap 
at  the  headquarters  of  any  large,  complex  organization. 

These  functions  could,  therefore,  be  applied  to  systems- 
engineering  teams  operating  within  the  DOD.  Specific  func¬ 
tions  that  these  teams  could  perform  in  implementing  the 
systems-engineering  methodology  on  the  service-headquarters 
level  and  within  OSD  are  outlined  below. 

Specific  Svstems-Engineering  Team  Functions  Within  OSD 

1.  To  assist  OSD  in  developing  an  overall  systems- 
acquisition-analysis  policy  for  DOD.  An  example  would  be  the 
development  of  the  methodology  outlined  in  the  previous 
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2.  To  assist  service  headquarters  in  implementing 
systems-acquisitions-analysis  policies  and  methodologies  to 
insure  uniformity  in  implementing  the  procedures  of  the 
methodology. 

3.  To  assist  the  service  headquarters  in  preparing 
for  specific  DSARC  major-decision  milestones. 

4.  To  assist  in  the  education  of  the  DSARC  members 
in  the  theory,  procedures,  and  practices  of  the  systems- 
engineering  methodology. 

5.  To  assist  in  the  development  of  the  actual  analysis 
structure  for  the  DSARC  program  evaluation,  selection,  and 
approval.  This  should  include  identifying  the  criteria  and 
factors  necessary  for  an  evaluation  of  the  candidate  system 
alternatives  and  the  overall  program.  Assisting  DSARC  mem¬ 
bers  in  developing  their  own  value  and  utility  functions  to 
evaluate  the  program  could  also  be  included. 

6.  To  assist  the  DSARC  members  in  carrying  out  the 
actual  trade  offs  involved  in  evaluating  the  major  systems- 
acquisition  programs,  including  how  various  operational 
policy  matters  could  be  revised  as  a  function  of  the  acquisi¬ 
tion  of  new  Defense  systems. 

Specific  Functions  of  the  Systems-Engineering  Team 
Within  Service  Headquarters 

1.  To  assist  the  system-program  offices  and  subordi¬ 
nate  commands  in  implementing  the  system-engineering 
methodology  for  systems  acquisition. 
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2.  To  assist  the  systems-program  offices  in  preparing 
for  specific  (S)SARC  major-decision  milestones. 

3.  To  assist  in  educating  the  (S)SARC  members  in  the 
theory,  procedures,  and  practices  of  the  systems-engineering 
methodology. 

4.  To  assist  in  developing  the  actual  analysis  struc¬ 
ture  for  the  (S)SARC  program  evaluation,  selection,  and 
approval . 

5.  To  assist  the  (S)SARC  members  in  carrying  out  the 
actual  trade  offs  involved  in  evaluating  the  major  systems- 
acquisition  programs. 

The  systems-engineering  teams  described  above  are  not 
required  in  every  organization  or  at  every  level  of  activity. 
However,  as  previously  indicated,  these  teams  may  be  effec¬ 
tively  employed  at  the  headquarters  of  large,  complex  organi¬ 
zations  that  have  multiple-analysis  activities  at  various 
locations.  In  the  context  of  the  DOD  environment,  systems- 
engineering  teams  should  be  established  within  OSD  and  within 
the  several  service  headquarters.  All  these  locations  are 
the  headquarters  of  large,  complex  organizations  with  multiple- 
analysis  activities  at  several  lower  echelons  and  at  a  variety 
of  locations.  Figure  32  graphically  depicts  the  location  of 
the  several  suggested  systems-engineering  teams  in  the  DOD 
environment.  The  Marine  Corps  does  not  normally  have  systems- 
program  offices  per  se  but  it  does  have  subordinate  commands 


Figure  3 
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that  analyze  systems  acquisitions  and  it  does  use  the  capa¬ 
bilities  of  the  other  services'  systems-program  offices. 

Each  of  the  service  headquarters  and  OSD  have  major 
staff  sections  primarily  concerned  with  research,  develop¬ 
ment,  and  acquisition.  The  systems-engineering  teams  could 
be  located  in  these  major  staff  sections.  To  insure  commu¬ 
nication  between  the  team  and  the  policymakers  or  decision 
makers,  the  manager  of  the  systems-engineering  team  should 
report  directly  to  the  senior  official  managing  the  staff 
section. 

Redundancy,  feasibility,  and  validity  of  requirement 
are  questioned  when  the  suggestion  is  made  to  establish  yet 
another  analysis  unit.  However,  there  is  no  current  stand¬ 
ardized  DOD  systems-engineering  methodology  to  assist  in  the 
decision  process  for  major  systems  acquisition.  In  addition, 
no  analysis  unit  in  the  several  locations  noted  above  is  per¬ 
forming  the  full  set  of  functions  described  for  the  systems- 
engineering  teams.  Therefore,  such  teams  would  not  contribute 
to  the  redundancy  of  functions. 

The  current  research  should  substantiate  the  validity 
of  systems-engineering  teams  of  the  type  described  above. 

The  work  on  the  structuring  of  the  systems-acquisition  pro¬ 
cess  and  the  development  of  the  methodology  show  the  possi¬ 
bilities  of  applying  systems  engineering  to  the  overall 
program.  Those  familiar  with  the  problems  of  communications 
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between  managers  and  analysts  should  also  see  the  benefit  of 
the  functions  outlined  for  the  systems-engineering  team. 

The  question  of  feasibility  is  best  addressed  by  con¬ 
sidering  the  activities  of  the  Headquarters  Department  of  the 
Army.  The  Directorate  of  Systems,  Review  and  Analysis, 
within  the  Office  of  the  Deputy  Chief  of  Staff  for  Research, 
Development,  and  Acquisitions,  Headquarters  Department  of  the 
Army  performs  some  functions  that  are  slightly  similar  to 
those  of  the  systems-engineering  team  (Army,  1978b) .  In 
addition,  the  Headquarters  Department  of  the  Army  has  pro¬ 
vided  for  independent  evaluation  teams  to  consider  specific 
problem  areas  relative  to  its  Army  SARC  major-decision  mile¬ 
stone  points.  These  independent  evaluation  teams  will 
develop  new  information,  "determine  counter-arguments,"  and 
"anticipate  specific  criticisms  from  sources  outside  the 
Army"  (Army,  1978b) .  Based  on  these  comments,  it  appears 
that  one  of  the  services  has  already  considered  implementing 
certain  functions  that  are,  at  least,  peripheral  to  those 
suggested  for  the  systems-engineering  team.  The  responsi¬ 
bilities  of  the  Office  of  the  Assistant  Secretary  of  Defense 
(Program  Analyses  and  Evaluation)  could  also  include  those 
outlined  for  the  systems-engineering  teams.  Therefore,  imple¬ 
mentation  of  the  systems-engineering  team  approach  should  be 
viewed  as  being  feasi  ,le  since  preliminary  activities  are 
already  operating  witnin  DOD. 


The  cost  of  implementing  the  systems-engineering 
methodology  and  the  systems-engineering  teams  would  be  mini¬ 
mal.  Some  slight  revisions  in  the  policy  dealing  with  systems 
acquisitions  and  a  number  of  procedural  changes  would  be 
necessary.  But,  the  actual  funding  required  would  be  pri¬ 
marily  in  the  area  of  the  qualified  systems-engineering  per¬ 
sonnel  and  the  necessary  technical  and  administrative  support. 
Smaller  systems-engineering  teams  of  four  to  seven  members 
could  address  a  specific  systems-acquisition  program  as  it 
approached  a  major-decision  milestone.  Normally  only  four  or 
five  programs  would  require  close  analysis  at  any  one  time. 
Therefore,  an  overall  systems-engineering  team  of  approxi¬ 
mately  twenty-five  people  at  each  of  the  major  headquarters 
could  implement  the  methodology  and  perform  the  functions 
previously  outlined. 

Effectiveness  Measures  for  Evaluating  the  Benefits  of 
Implementing  the  Systems-Engineering  Methodology 

The  final  topic  of  this  section  covers  the  following 
measures  of  effectiveness  for  evaluating  the  benefits  of 
implementing  the  methodology  and  forming  the  systems- 
engineering  teams. 

1.  The  number  of  MENS  developed,  processed,  and 
approved  for  active  programs  should  increase.  If  the  re¬ 
quirements  for  the  MENS  are  more  clearly  defined,  then  they 
will  be  more  readily  completed.  This  will  enable  all  pro¬ 
grams  to  be  on  equal  footing,  thereby  insuring  a  greater 
degree  of  equity  in  the  evaluation  and  selection  process. 
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2.  The  number  of  areas  where  operational  requirements 
are  ambiguous  should  be  reduced.  More  clearly  defined  MENS 
should  result  from  implementation  of  the  methodology  and  its 
problem-definition  aspects.  More  clearly  defined  problem 
statements  should  reduce  areas  of  conflict  between  and  dupli¬ 
cation  of  requirements. 

3.  The  overall  projected  funding  for  the  completion 
of  all  programs  should  be  reduced.  The  absolute  number  of 
programs  should  be  reduced  with  the  introduction  of  proce¬ 
dures  to  trade  off  the  various  programs  early  in  the  acquisi¬ 
tion  process.  This  reduction  should  also  lead  to  a  reduction 
in  projected  overall  funding. 

4.  The  variance  among  the  conditions  of  programs 
appearing  before  the  DSARC/ (S) SARC  should  be  reduced.  The 
methodology  will  identify  key  evaluation  factors  critical  to 
each  state  of  the  decision  process.  The  highlighting  of 
these  factors  should  lead  to  more  consistent  program  perfor¬ 
mance.  For  example,  this  variance  could  be  measured  at  the 
production-decision  milestones  by  measuring  the  number  of 
Engineering  Change  Proposals  per  million  dollars  of  produc¬ 
tion  effort  for  each  program.  Another  quantifiable  measure  of 
this  variance  in  the  logistical  support  area  is  the  number  of 
new  supply-line  items  introduced  to  the  overall  inventory  per 
million  dollars  of  production  effort  for  each  program. 
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5.  The  number  of  programs  failing  or  requiring  major 
revisions  in  the  phase  after  a  specific  decision  point  should 
be  reduced.  As  the  methodology  is  implemented  and  improved, 
a  greater  consistency  will  develop  between  program  approval 
and  program  performance.  As  the  methodology  evolves,  the 
decision  makers  should  be  better  able  to  determine  which  key 
evaluation  factors  are  the  best  predictors  of  future  program 
performance.  This  would,  of  course,  be  a  long-range  means  of 
evaluating  the  programs  and  the  implementation  of  the 
methodology. 


Summary  and  Conclusions 

This  chapter  discussed  the  various  aspects  of  develop¬ 
ing  a  systems-engineering  methodology  for  Defense  systems 
acquisition.  The  methodology  is  designed  to  support  the 
DSARC/ ( S ) SARC  decision  process  through  systems  engineering. 

The  methodology  is  specifically  directed  toward  providing 
analysis  support  for  major-decision  Milestones  0,  I,  II,  and 
III  of  the  systems-acquisition  process. 

Initially,  the  existing  procedures  associated  with  the 
decision  process  and  value  system  operative  within  the  DSARC 
and  several  (S)SARCs  were  examined.  The  nature  and  structure 
of  the  Defense  systems-acquisition  process  were  then  investi¬ 
gated.  The  criteria  and  organizational  considerations  for 
applying  the  systems-engineering  tools  and  techniques  were 
identified.  Next,  the  appropriate  emphasis  of  the  methodology 
was  considered.  This  resulted  in  identifying  several  areas 


where  the  systems-engineering  analysis  would  best  support 
the  DSARC/ (S) SARC  decision  process.  The  specific  analysis 
procedures  to  be  used  in  the  methodology  were  then  outlined 
and  associated  with  each  of  the  decision-process  support 
areas  previously  identified.  Finally,  various  methodology 
implementation  considerations  were  discussed.  This  section 
included  a  discussion  of  the  motivation  for  and  the  functions 
and  characteristics  of  several  systems-engineering  teams 
projected  to  support  the  implementation  of  the  methodology. 

Based  on  the  material  developed  in  this  chapter,  the 
following  conclusions  are  drawn: 

1.  That  there  is  a  perceptible  value  system  opera¬ 
tive  within  the  OSD  and  the  several  service  headquarters 
relative  to  the  Defense  systems-acquisition  decision  process. 

2.  That  a  study  of  the  policies  and  procedures  asso¬ 
ciated  with  the  systems-acquisition  process  leads  to  the 
following  significant  observations: 

a. .  That  the  MENS  development  phase  is,  in  reality, 
the  problem-definition  step  of  the  systems-acquisition  proc¬ 
ess  and,  as  such,  is  critical  to  the  overall  analysis  effort. 

b.  That  it  is  important  to  consider  possible 
means  by  which  the  candidate  programs,  as  represented  by  the 
MENS,  could  be  traded  off  early  in  the  systems-acquisition 
process . 

c.  That  the  decisions  required  at  each  major- 
decision  milestones  are  different  in  their  nature,  intent, 
and  impact. 
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3.  That  the  systems-acquisition  process  may  be  modelled 
as  a  stage  approach  problem  with  normally  decreasing  uncer¬ 
tainty,  increasing  fund  requirements,  and  increasing  availa¬ 
bility  of  reliable,  quantitative  data  as  the  process  advances 
from  phase  to  phase  (or  stage  to  stage) . 

4.  That  organizational  considerations  are  important 
in  determining  the  type,  nature,  and  degree  of  the  analysis 
effort  to  be  applied  within  a  systems-engineering  methodology. 

5.  That  the  following  areas  associated  with  the 
DSARC/ (S) SARC  decision  process  could  benefit  by  analysis 
within  a  systems-engineering  methodology: 

a.  The  establishment  of  procedures  for  developing 
a  MENS  that  would  more  clearly  define  the  operational- 
requirements  problems  and  make  its  characteristics  more 
identifiable. 

b.  The  evaluation  procedure  by  which  the  programs, 
as  represented  by  the  MENS,  might  be  traded  off  early  in  the 
acquisition  cycle. 

c.  The  analysis  procedures  which  support  the 
DSARC/ (S) SARC  decision  process  at  major-decision  Milestones  I, 
II,  and  III. 

6.  That  the  several  analysis  procedures  outlined  as 
tasks  for  the  methodology  are  general  steps  or  specific 
phases  within  the  systems-engineering  morphology. 
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7.  That  the  effective  implementation  of  the  metho¬ 
dology  could  be  significantly  enhanced  by  the  formation  and 
proper  use  of  systems-engineering  teams  working  with  top- 
level  managers  within  OSD  and  the  several  service  headquarters. 

The  analysis  procedures  previously  outlined  as  tasks 
within  the  systems-engineering  methodology  do  not  represent  a 
comprehensive  application  of  all  possible  systems-engineering 
tools  and  techniques  to  support  the  systems-acquisition  de¬ 
cision  process.  Rather,  the  analysis  tasks  are  representa¬ 
tive  of  the  type  of  systems-engineering  support  that  could 
be  provided  to  the  decision  process.  Further  research  into 
the  decision  process  should  identify  other  areas  where  addi¬ 
tional  systems-engineering  tools  and  techniques  could  be 
beneficially  applied. 

The  next  chapter  presents  a  real-world  example  of  the 
application  of  the  systems-engineering  approach  to  a  Defense 
systems-acquisition  problem.  This  example  specifically 
adresses  the  formation  and  use  of  a  systems-engineering  team 
trying  to  bridge  the  gap  discussed  in  the  Implementation  of  the 
Methodology  section  of  this  chapter. 
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Chapter  5 


RESEARCH  INTO  AN  APPLIED  SYSTEMS-ACQUISITION  PROBLEM 

Introduction 

A  strategy  has  been  documented  in  previous  chapters 
for  using  the  systems-engineering  methodology  to  support  the 
Defense  systems-acquisition  decision  process  .  The  need  for 
systems-engineering  teams  to  assist  in  implementing  the 
methodology  has  been  documented  and  the  functions  and  charac¬ 
teristics  associated  with  these  teams  have  been  outlined. 

The  specific  methodology  outlined,  or  other  possible  approaches, 
would  be  most  effectively  implemented  by  developing  and  properly 
using  the  suggested  systems-engineering  teams.  To  demonstrate 
this,  an  applied  problem  is  investigated  from  a  historical 
perspective.  Specifically,  a  Defense  systems-accuisition 
program  is  examined  to  see  what  impact  the  employment  of  a 
systems-engineering  team  and  approach  would  have  on  the  develop¬ 
ment  effort. 

The  DOD  development  effort  selected  for  examination 
is  a  joint  systems-acquisition  program  between  the  Air  Force 
and  the  Marine  Corps.  The  overall  program  is  oriented  toward 
developing  a  computer-based  system  for  the  processing  of 
tactical  intelligence  and  was  outlined  in  an  Air  Force  systems 
command  document  (Air  Force,  1978) .  Specific  Marine  Corps 
involvement  in  the  program  was  directed  by  Marine /Air  Ground 


Intelligence  System  (MAGIS)  (Navy,  1978).  An  overview  of 
how  the  system  should  operate  in  an  operational  environment 
was  presented  by  Layne  and  Jorgenson  (1974) . 

The  overall  DOD  name  and  the  Air  Force's  specific 
name  for  the  system  is  the  Tactical  Information  Processing 
and  Interpretation  (TIPI)  System;  the  Marine  Corps  calls  it 
the  Marine  Air/Ground  Intelligence  System  (MAGIS) .  MAGIS 
consists  of  several  subsystems,  or  segments,  which  are 
addressed  in: 

1.  R&D  Status  Report  and  Program  Schedule 3  TIPI 
System  Integration  and  Checkout  Contract,  (GE,  1979). 

2.  Marine  Air /Ground  Intelligence  System  (MAGIS) 
Development  and  Operational  Concepts,  (Navy,  1973c). 

3.  Systems  Hardware  and  Software  Characteristics 
for  the  Marine  Air /Ground  Intelligence  System  (MAGIS) , 

(Navy,  1975d) . 

Development  of  the  system  has  been  going  on  since  1965. 
Developmental  and  operational  testing  of  two  of  the  system's 
segments  has  been  reported  by: 

1.  Testing  of  Complex  Man/Machine  Intelligence  Systems 
A  Case  Study  Perspective  of  TIPI  (MAGIS)  Past ,  Present }  and 
Future,  (Harrison,  1976). 

2.  Segment  Test  Plan  for  the  Intelligence  Analysis/ 
Storage  and  Retrieval  Category  II  Test  Segments  of  the  TIPI 
System /MAGIS ,  (Systems  Development  Corporation,  1974). 
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3.  Final  Report  of  the  Imagery  Interpretation  (II) 
Segment  Development  Test  and  Evaluation/Initial  Operational 
Test  and  Evaluation  for  the  TIPI/MAGIS  System,  (Texas 
Instruments,  1975) . 

The  projected  operational  employment  concepts  for  the 
system  when  it  is  fielded  have  undergone  an  evolutionary 
process  during  the  system  development  (GE,  1972,  1974a; 

Navy,  1973b).  This  evolution  of  the  Marine  Corps'  operational 
employment  concepts,  coupled  with  a  greater  understanding  of 
the  overall  systems  problem  as  indicated  by  Utilization  of 
Modular  Equipment  within  the  MAGIS  Development  Program,  (Navy, 
1973e) ,  led  to  a  major  revision  in  the  systems-acquisition 
approach  outlined  in  Request  for  Change  to  Program  Memorandum 
§50,  (Navy,  1975c).  The  development-program  modifications 
necessary  to  implement  the  revised  employment  concepts  and 
systems-acquisition  approach  were  affected  by  policy  changes 
within  both  the  Air  Force  and  Marine  Corps  (Air  Force,  1975; 
Navy,  1975c) . 

The  problems  of  the  systems-development  program  leading 
to  the  revision  of  the  Marine  Corps'  employment  concepts, 
the  analysis  of  these  problems,  and  the  headquarters-level 
decisions  to  revise  the  acquisition  approach  based  on  the 
analysis  are  the  key  elements  of  the  problem  studied  here. 

The  development  and  use  of  a  systems-engineering  team  to 
address  the  systems-acquisition  program  was  essential  to  the 
resolution  of  this  problem.  The  team  specifically  assisted 
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the  Marine  Corps'  top-level  management  in  preparing  for  the 
major-decision  Milestone  II.  Studying  the  development  effort 
associated  with  the  MAGIS  acquisition  program  may  motivate 
the  employment  of  such  a  team  in  research  and  development  (R&D) 
Therefore,  the  following  three  subjects  will  be  covered  in 
this  chapter: 

1.  The  background  of  the  systems-development  program. 

2.  The  period  in  development  when  the  program  suffered 
from  the  lack  of  a  systems-engineering  approach. 

3.  The  period  in  the  development  effort  when  the 
program  benefited  from  systems-engineering  approach. 

There  are  several  reasons  why  the  MAGIS  development 
program  has  been  selected  as  an  example.  First,  from 
February,  1972  to  August,  1975,  the  author  was  the  Development 
Project  Officer  for  this  joint  development  program  and  the 
senior  Marine  officer  working  full-time  on  the  program. 

Second,  the  program  is  a  large,  complex,  and  costly  ($300 
million  cumulative  total)  program  that  now  involves  all  four 
services.  Third,  the  documentation  to  carry  out  the  analysis 
effort  is  readily  available.  Fourth,  and  most  important,  a 
systems-engineering  team,  operating  in  the  mode  discussed  in 
the  previous  chapter,  was  employed  to  assist  the  top-level 
decision  makers  in  the  Marine  Corps  in  addressing  a  Defense 
systems-acquisition  problem. 
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Background  of  the  Sy stems-Development  Program 


The  Marine  Air/Ground  Intelligence  Systems  (MAGIS)  is 
being  developed  as  a  tactical  intelligence-processing  system 
for  use  by  the  Marine  Air/Ground  Task  Forces  (MAGTF)  in 
amphibious  warfare.  The  Assistant  Chief  of  Staff,  Intelligence 
(G-2)  of  the  amphibious  landing  force  is  responsible  for 
providing  the  intelligence  needed  for  command  decisions. 

To  discharge  his  responsibilities,  the  G-2  must  be  able  to 
produce  accurate,  complete,  timely,  and  useful  intelligence 
during  each  phase  of  the  amphibious  operation.  MAGIS  is 
being  developed  to  provide  this  capability  through  automated 
assistance  to  combat-intelligence  activities.  This  will 
include  automated  assistance  to  what  has  historically  been 
considered  the  special  (i.e.,  electronics)  intelligence 
function.  The  inclusion  of  this  function  will  make  the  system 
all-source^  and  enable  it  to  integrate  data  from  a  wide-range 
intelligence  collection  means. 

The  system  will  be  capable  of  being  transported  by 
air,  ground,  and  sea.  It  will  be  housed  in  a  set  of  8'X  8'X  20' 
steel  shelters  with  a  modern  array  of  communications  and  data- 
processing  capabilities.  The  incentive  for  developing  a 
a  system  of  this  type  was  an  imbalance  that  developed  between 
the  Marine  Corps'  information-collection  capability  and  the 
intelligence-processing  capability.  In  recent  years  the 


^ All-source  is  the  ability  to  receive  all  types  of  intelligence 
data,  including  special  intelligence. 
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Marine  Corps'  information-collection  capabilities  have  been 
significantly  increased  by  the  use  of  remote  sensors, 
aerial  reconnaissance,  and  the  monitoring  of  selected 
enemy  electromagnetic  radiations  (Layne  and  Jorgenson,  1974). 
During  the  same  period  of  time,  the  Marine  Corps'  ability  to 
process,  evaluate,  and  analyze  information  has  remained  rela¬ 
tively  constant.  These  factors  led  to  an  imbalance  between 
the  collection  capabilities  and  the  processing  capabilities 
where  in  the  inputs  produced  by  the  former  were  restricted 
as  throughput  by  the  latter.  The  goal  of  the  system  develop¬ 
ment  is  to  correct  this  imbalance  and  to  support  the  amphibious  - 
landing -force  commander  and  subordinate  commanders  in  their 
decision  making  processes. 

In  1963,  the  Commandant  of  the  Marine  Corps  (Navy,  1963) 
directed  the  development  of  an  integrated  air/ground  intelli¬ 
gence  system  to  be  known  as  MAGIS.  At  the  direction  of  DOD 
(Defense,  1964)  a  joint  service  program  under  the  executive 
management  of  the  Air  Force  was  established  to  develop  a 
tactical  intelligence-processing  system  for  all  four  services. 
Initially  the  Army  and  Navy  simply  monitored  the  development 
effort,  but  in  recent  years,  they  have  become  more  closely 
associated.  The  Army  will  participate  in  the  production  buy 
for  one  of  the  systems'  segments  (Air  Force,  1978),  and  the 
Navy  will  use  the  software  designed  for  another  segment  to 
replace  its  existing  Naval  Intelligence  Process  System  aboard 
certain  classes  of  its  ships  (Navy,  1979)  .  The  program  remains 
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a  joint  effort  under  Air  Force  executive  management  with 
personnel  and  R&D  funding  being  contributed  by  the  Marine 
Corps  and  the  Air  Force.  Currently  the  Development  Center, 
Marine  Corps  Development  and  Education  Command,  Quantico, 
Virginia,  represents  the  Marine  Corps  in  the  development 
effort  and  is  the  contact  point  for  all  matters  concerning 
MAG IS. 

The  classical  intelligence  cycle  is  made  up  of  four 
phases:  direction,  collection,  processing,  and  dissemination 
(see  DELTA  chart ,  Figure  33.  MAGIS  will  support  several  segments  of 
this  development  cycle  with  the  necessary  maintenance  and  logisti¬ 
cal  capabilities;  however,  the  program  emphasis  for  MAGIS  is 
to  develop  a  system  to  upgrade  the  processing  of  intelligence. 
Certain  benefits  will  also  be  derived  from  the  system  for 
other  phases  of  the  intelligence  cycle.  Currently,  the 
Marine  Corps  has  various  collection  assets2  to  produce  inputs 
for  a  manual  processing  system.  The  various  stages  of  pro¬ 
cessing  result  in  finished  intelligence  products,  such  as, 
estimates  of  enemy  capabilities  and  summaries  of  previous 
enemy  activities.  MAGIS  should  upgrade  the  existing  manual 
techniques  for  processing,  thereby  substantially  increasing 
the  effectiveness  of  intelligence  production. 

2 

Collection  assets  are  the  capabilities  by  which  raw  information 
is  collected  from  the  battlefield.  Examples  of  collection  assets 
are  tactical  units  in  contact  with  the  enemy,  electronic  monitors, 
reconnaissance  aircraft,  and  infrared  sensors  (Navy,  1974c).. 


275 


The  nature  of  the  overall  intelligence  cycle  may  be 
better  understood  if  it  is  viewed  as  an  input-output  process. 

In  this  context,  the  collection  effort  provides  the  inputs 
(raw  information) ;  the  finished  intelligence  is  produced 
in  the  processing  phase  and  distributed  in  the  dissemination 
phase.  The  overall  process  is  graphically  presented  in 
Figure  34.  This  figure  portrays  the  collection  of  raw 
information  and  the  processing  of  this  information  by  all¬ 
source  intelligence  analysts  to  produce  finished  intelligence 
products . 

The  Period  in  Development  When  the  Program  Suffered 
from  the  Lack  of  a  Sy s terns -Eng ineering  Approach 

In  1964,  the  DOD  designated  the  Air  Force  the  executive 
agent  for  the  joint  service  program  and  established  the 
Tactical  Information  Processing  and  Interpretation  Program 
Office  (TIPI  PO)  to  manage  the  program.  The  TIPI  PO  is 
part  of  the  Electronic  System  Division  of  the  Air  Force 
Systems  Command,  located  at  L.  G.  Hanscom  Field,  Bedford, 
Massachusetts.  The  systems-program  office  received  policy 
guidance  from  Headquarters  United  States  Air  Force  and  opera¬ 
tional  requirements  from  the  Air  Forces'  Tactical  Air  Command. 
Based  on  its  designation  as  the  DOD  system-program  office, 

TIPI  PO  could  directly  manage  and  execute  the  development 
program,  thus  making  it  the  focal  point  for  the  Air  Force's 
involvement  in  the  joint  service  program. 
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In  the  early  years  of  the  program,  the  Marine  Corps' 
involvement  in  the  program  was  somewhat  more  diffused  than 
the  Air  Force's.  Both  Headquarters  Marine  Corps,  and 
the  Development  Center,  Marine  Corps  Development  and  Education 
Command  (MCDEC)  were  involved  in  managing  the  Marine  Corps' 
participation  in  the  effort.  Operational  requirements  evolved 
from  the  Development  Center  and  the  Marine  Corps  Operational 
Commands.  Figure  35  graphically  portrays  this  situation 
prior  to  1972  with  interactions  between  various  Marine  Corps 
activities  and  the  TIPI  program  office.  Partially  because  of 
this  diffusion  of  responsibilities,  several  different  subsystems 
or  segments  were  identified  in  the  early  1970 's  as  separate 
operational  requirements  under  the  overall  MAG IS  umbrella 
(Navy,  1978).  The  full  titles  of  these  segments,  their 
acronyms,  and  a  brief  description  of  their  functions  are 
presented  in  Table  10. 

The  first  two  segments  noted  in  the  table.  Imagery 
Processing  and  Tactical  Electronic  Reconnaissance  Processing 
and  Evaluation,  were  to  be  up-grades  of  existing  capabilities. 
The  third  segment,  Imagery  Interpretation,  was  newly  developed 
to  analyze  imagery  from  the  services'  photographic  reconnais¬ 
sance  aircraft. 

The  final  three  functional  segments  listed  in  the  table, 
the  Remote  Input/Output  System  (RIOS) ,  the  Signal  Intelli¬ 
gence/Electronic  Warfare  Coordination  Center  (S/EWCC) ,  and 
the  Intelligence  Analysis/Storage  and  Retrieval  (IA/SR) ,  are 
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TABLE  10 

FUNCTIONAL  SEGMENTS  OF  THE  MARINE  AIR/GROUND 
INTELLIGENCE  SEGMENTS  IN  1972 


Long  Title 

Acronym 

Function 

Imagery  processing 

IP 

Development  and  duplication 
of  photographic  materials 

Tactical  electronic  recon¬ 
naissance  processing  and 
evaluation 

TERPE 

Processing  capability  for 
the  identification  and 
location  of  enemy  elec¬ 
tronic  emitters 

Imagery  interpretation 

II 

Analysis  of  imagery  for 
intelligence  purposes 

Remote  input/output 
system 

RIOS 

Processing  of  all  intelli¬ 
gence,  except  special,  at 
the  Marine  division  and 
wing  levels 

Signal  Intelligence 
Electronic  Warfare 
Coordination  Center 

S/EWCC 

Special  intelligence  produc¬ 
tion  and  signal  intelli¬ 
gence  electronic  warfare 
coordination  at  the  Marine 
amphibious-force  level 

Intelligence  analysis, 
storage,  and  retrieval 

IA/SR 

Processing  of  all  intelli¬ 
gence  ,  except  special ,  at 
the  Marine  amphibious- 
force  level 
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of  the  most  interest  in  the  context  of  this  historical 
analysis.  The  RIOS  was  intended  to  provide  an  intelligence¬ 
processing  capability  to  the  Marine  Division  and  Marine 
Aircraft  Wing  levels  (Navy,  1972c) .  The  S/EWCC  was  intended 
to  provide  a  special  (i.e.,  electronics)  intelligence¬ 
processing  capaibility  to  the  Marine  Amphibious  Force 
level  (the  overall  headquarters  for  the  wing  and  division 
team) (Navy,  1970) .  One  of  the  principal  factors  motivating 
this  segment  was  the  Marine  Corps  operational  policy  require¬ 
ment  to  separate  special  intelligence  from  other  types  of 
intelligence.  The  IA/SR  was  intended  to  provi.de  an 
intelligence-processing  capability  to  the  Marine  Amphibious 
Force  level  (Navy,  1969a;  Navy,  1969b)  .  All  three  of  these 
segments  were  to  have  similar  functions  related  to  processing 
of  intelligence;  only  the  type  of  data  and  the  command  level 
at  which  they  operated  were  to  differ.  The  S/EWCC  and  RIOS 
were  unique  Marine  Corps  operational  requirements  in  the  over¬ 
all  systems-development  program. 

In  1972,  all  three  of  these  segments  were  under 
development,  but  each  in  a  different  phase  of  the  systems- 
acquisition  process  and  each  by  a  different  contractor. 

The  RIOS  was  in  the  requirements-def inition  (mission-analysis) 
phase  at  General  Electric  Company  in  Daytona  Beach,  Florida; 
the  S/EWCC  was  in  the  conceptual  phase  at  the  Bunker  Ramo 
Corporation  in  West  Lake  Village,  California;  and  the  IA/SR 
was  in  full-scale  development  at  the  Systems  Development 
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Corporation  in  Hampton,  Virginia.  This  final  segment  was 
scheduled  for  development,  test,  and  evaluation  (DT&E)  and  for 
initial  operational  test  and  evaluation  (IOT&E)in  1975.  RDT&E 
funds  of  $30  million  had  been  committed  to  this  program  to 
develop,  test,  and  evaluate  one  preproduction  model  for 
the  Air  Force  and  one  for  the  Marine  Corps.  In  accordance 
with  a  memorandum  of  agreement  between  the  U.  S.  Air  Force 
and  the  Marine  Corps  (July,  197  3)  ,  the  Marine  Corps  provided 
a  prorated  share  of  these  funds  amounting  to  $6.5  million. 
However,  newly  identified  operational  requirements  (i.e., 
compatibility  with  the  Navy's  tactical-intelligence  data 
base)  and  essential  software  improvements  necessitiated  an 
additional  expenditure  of  $1.2  million  in  funding  for  this 
segment.  The  3/EWCC  and  the  RIOS  had  a  combined  total  of 
$4.5  million  tentatively  programmed  for  the  development  of 
preproduction  models  to  be  used  for  DT&E/IOT&E  (Navy,  1972b) . 
There  was  significant  concern  within  the  Development  Center 
of  the  Marine  Corps  Development  and  Education  Command  that  the 
programmed  funds  for  the  S/E17CC  and  RIOS  segments  would  be 
grossly  insufficient  to  accomplish  the  projected  development. 

A  conservative  estimate  of  the  actual  funds  required  was 
$5  million  for  each  segment,  for  a  combined  total  of 
$10  million."3 

3This  estimate  was  based  on  the  Marine  Corps'  contribution  to 
the  IA/SR  preproduction-model  (full-scale)  development ,  which  had  cost  little 
more  than  the  systems  hardware  and  had  not  included  such  things  as  the 
engineering  design  and  software-development  efforts.  Later  analysis  efforts 
within  the  development  center  of  MCDEC  projected  RDT&E  costs  of  $4.8  million 
and  $8.5  million  for  the  RIOS  and  5/EWCC  segments,  respectively  (Navy,  1973a) . 
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During  1972  the  overall  status  of  the  MAG IS  development 
program  was  somewhat  uncertain.  There  were  three  contractors 
in  three  separate  locations  carrying  out  development  efforts 
for  three  different  segments.  Each  development  effort  was 
intended  to  provide  the  Marine  Corps  with  approximately 
the  same  type  of  functional  and  technical  capabilities  to 
process  tactical  intelligence.  The  potential  problems 
listed  below  were  inherent  in  carrying  out  parallel  develop¬ 
ments  to  acquire  basically  the  same  type  of  operational 
capability . 

1.  The  development  plans  during  this  period  projected 
the  expenditure  of  substantial  funds  for  what,  in  many  respects, 
was  redundant  development. 

2.  The  hardware  components  for  each  segment  would, 
most  likely  vary  considerably,  which  would  require  several 
different  maintenance  and  support  approaches  for  the  system. 

The  Marine  Corps  did  not  have  sufficient  resources  to  support 
several  different  approaches  once  the  system  was  deployed. 

3.  The  software  developed  for  each  segment  would 
probably  be  different,  requiring  multiple  training  programs 
for  system  operators  or  different  operators  for  each  segment. 
Either  alternative  was  incompatible  with  the  Marine  Corps ' 
constrained  manpower  resources. 

4.  The  separate  development  of  three  parallel  seg¬ 
ments  made  interoperability  problems  for  the  overall  system 
inevitable  once  the  system  was  fielded.  For  example,  the 
intelligence  data  base  and  the  communication  protocols  for 
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the  segments  differed,  precluding  the  interchange  of  the 
information  the  system  was  being  developed  to  process. 

5.  The  development  of  so  many  different  segments, 

(six  are  listed  in  Table  10)  needed  a  global  operational 
concept  for  fielding  the  overall  system.  The  lack  of  these 
operational  concepts,  which  did  not  exist  in  1972,  foretold 
of  significant  problems  when  the  Marine  Corps’  combat  units 
in  the  field  would  have  to  employ  the  system. 

6.  The  development  of  three  parallel  segments  and 
the  substantial  RDT&E  funds  ($10  million  would  be  a  conserva¬ 
tive  estimate)  required,  implied  that  top-level  decision  makers 
in  the  Marine  Corps  were  not  fully  cognizant  of  the  overall 
systems-acquisition  process  associated  with  the  MAG IS  program. 
Ultimately,  the  lack  of  understanding  or  top-level  management 
support  could  have  led  to  the  failure  of  the  program. 

The  means  by  which  the  above-mentioned  potential 
problems  were  addressed  is  discussed  in  the  next  section. 

The  Period  in  Development  When  the  Program  Benefited 
from  a  Sy stems-Engineering  Approach 

In  the  previous  section,  the  problem  of  the  diffusion 
of  development  responsibi] ities  for  the  MAGIS  program  within 
the  Marine  Corps  was  discussed.  During  1972  this  problem  was 
corrected  by  Headquarters  Marine  Corp  ( HQMC ) .  The  fiscal 
year  1973  (FY-1973)  RDT&E  VTork  Directive  for  Magis  (Navy,  1972b) 
stated,  in  part,  that 
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The  Commanding  General,  Marine  Corps  Development  and  Educa¬ 
tion  Command  (MCDEC) ,  Quantico,  Virginia,  will  manage  all  aspects 
of  the  MAGIS  development. . .on  matters  relating  to  Commandant  of  the 
Marine  Corps  (CMC)  policy,  program  funding,  and  operational  system 
developments,  the  Commanding  General,  MCDEC,  will  obtain  CMC 
guidance  and/or  concurrence. 

This  directive  clearly  gave  the  responsibility  for  managing 
the  MAGIS  acquisition  program  to  the  Commanding  General  of 
MCDEC  except  in  those  areas  where  the  decision  process 
touched  upon  CMC  policy  matters.  This  management  responsi¬ 
bility  included  planning  and  controlling  the  R&D  funds  for 
the  program.  Figure  36  portrays  this  new  situation  with 
MCDEC  as  the  focal  point  for  MAGIS  development.  Management 
and  operational  requirements  interactions  with  HQMC  and  the 
Fleet  Marine  Force,  respectively,  were  conducted  through 
MCDEC  to  the  Air  Force  TIPI  Program  Office. 

Within  the  Marine  Corps  Development  and  Education 
Command,  the  Development  Center  and  specif ically ,  the  Intelli¬ 
gence  Division  of  the  Development  Center  was  asked  to  carry 
out  the  MAGIS  acquisition  program.  A  separate  branch,  the 
Automated  Intelligence  Systems  3ranch,  was  established  within 
the  Intelligence  Division  of  the  Development  Center  to  address 
the  MAGIS  program.  Initially  this  branch  consisted  of  the 
branch  head  (or  MAGIS  Project  Officer)  and  two  other  people, 
none  of  whom  had  been  formally  educated  in  systems  engineering 
or  had  previously  worked  in  a  systems-engineering  group.  The 
need  to  develop  a  multidiscipline  engineering  team  to  address 
the  MAGIS  acquisition  program  was  identified  in  October  1972. 

l 


Figure  36.  Centralization  of  Development  Responsi 
bilities  within  the  Marine  Corps  for  the  MAGIS  Systems- 
Acquisition  Program  after  1972. 
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This  need  was  documented  by  the  Chief  of  the  Intelligence 
Division  of  the  Development  Center  (Navy,  1972a)  and  stated, 

The  Development  Center  is  currently  involved  in  a  highly 
technical  program  for  the  development  of  MAGIS.  This  develop¬ 
ment  effort  cuts  across  a  multitude  of  technical  disciplines . 

However,  the  technical  expertise  needed  to  most  effectively 
address  and  solve  the  developmental  problems  is  not  presently 
available  for  allocation  against  the  technical  R&D  require¬ 
ments  ....  [Therefore]  it  is  recommended  that  the  Development 
Center  take  appropriate  action  to  establish  an  engineering 
support  group  for  the  development  of  MAGIS. 

In  November,  1972,  the  Director  of  Naval  Laboratories, 
Department  of  the  Navy,  the  Director  of  the  Development  Center, 
and  representatives  of  Headquarters  Marine  Corps  were 
briefed  on  the  need  for  multidiscipline-engineering  support 
for  the  MAGIS  program  (Navy,  1975b).  As  a  result,  two  civil 
service  technical  positions  (one  GS-14  electrical  engineer 
and  one  GS-13  computer  scientist)  were  established  for  the  MAGIS 
program  within  the  Intelligence  Division  of  the  Development 
Center.  Several  military  officer  positions,  requiring  a 
graduate  education,  were  also  reassigned  to  the  MAGIS  program. 
Ultimately,  the  multidiscipline  engineering  team  grew  to  a 
total  of  eight  people,  including  the  team  leader,  the  MAGIS 
Project  Officer. 

This  MAGIS  project  officer,  a  major  in  the  Marine 
Corps,  held  a  bachelor's  degree  in  mechanical  engineering  and 
a  Masters  in  operations  research  and  had  previous  management, 
operational,  and  analyst  experience.  He  reported  directly  to 
the  Chief  of  the  Intelligence  Division,  Development  Center, 
a  colonel  in  the  Marine  Corps.  The  team  leader  managed  the 
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overall  activities  of  the  team  and  had  full  authority,  in 
addressing  the  problems  associated  with  the  MAGIS  acquisi¬ 
tion  program,  to  allocate  the  resources  of  the  team.  The 
team  leader  and  the  team  v/ere  charged,  through  the  P.DT&E  Work 
Directive  for  MAGIS  (Navy,  1978),  with  the  following  respon¬ 
sibilities  , 

1.  Act  as  Marine  Corps  point  of  contact  for  all  TIPI- 
Program  Office  matters  of  interest  to  the  Marine 
Corps  concerning  the  development,  testing,  and 
reporting  of  the  TIPI/MAGIS  project. 

2.  Continuously  study  and  appraise  the  phases  of  MAGIS 
development,  procurement,  testing,  and  reporting  to 
include: 

a.  Maintain  close  direct  liaison  with  TIPI-Program 
Office  and  provide  the  necessary  operational 
guidance  to  insure  that  individual  members  of 
that  office  and  their  contractors  know  and 
understand  Marine  Corps  MAGIS  requirements, 

b.  Maintain  liaison  with  other  service  and  com¬ 
mercial  organizations  on  matters  relating 

to  TIPI/MAGIS  development, 

c.  Ensure  that  all  operational  requirements  and 
other  required  documentation  for  MAGIS  are  pro¬ 
vided  to  TIPI-Program  Officer. 

Five  members  of  the  team  had  graduate-level  education. 
Several  disciplines  v/ere  represented  on  the  team,  including 
electrical  engineering,  computer  science,  mathematics,  communi 
cations  engineering,  data-systems  management,  systems  analysis 
operations  research,  humanities  (English) ,  and  mechanical 
engineering.  All  members  of  the  team,  except  the  two  civil 
service  employees,  had  extensive  operational  experience  within 
the  Marine  Corps.  Seven  members  of  the  teem  had  previously 
held  managerial  positions  at  several  levels  in  a  variety  of 
organizations;  six  members  had  prior  P&D  experience;  and  three 


members  of  the  team  had  been  formally  trained  as  analysts. 
Comparing  the  characteristics  and  functions  outlined  in 
Chapter  4  for  systems-engineering  teams  and  the  characteris¬ 
tics  and  responsibilities  outlined  above  for  the  MAG IS  team, 
a  systems-engineering  team  had  obviously  been  formed  to 
address  the  MAGIS  acquisition  program. 

The  systems-engineering  team  for  the  MAGIS  program  was 
formed  within  the  Development  Center  of  the  Marine  Corps 
Development  and  Education  Command  located  at  Quantico, 

Virginia.  Following  the  suggestions  in  the  methodology 
implementation  section  (p.244)  of  Chapter  4,  the  systems- 
engineering  team  should  be  located  within  the  R&D-staff  section 
of  service  headquarters.  Because  the  Marine  Corps  Development 
Center  had  been  given  the  responsibility  to  manage  the  MAGIS 
program  and  limited  Marine  Corps  manpower  resources  -would 
not  permit  duplication  of  the  function  at  the  headquarters 
level,  the  MAGIS  systems-engineering  team  could  not  be 
located  as  recommended  in  Chapter  4.  However,  the  MAGIS 
systems-engineering  team  did  functionally  bridge  the  gap 
as  described  in  Chapter  4,  between  the  various  analysis  and 
engineering  groups  and  the  decision  makers  and  j-clicy  makers. 
The  team  was  able  to  have  an  impact  on  the  decision  making 
and  policy  making  process  at  the  level  of  Headquarters  Marine 
Corps.  This  team's  opportunity  to  act  in  the  role  came 
about  for  several  reasons.  First,  the  management  of  the 
Development  Center  gave  the  team  every  opportunity  to  interface 
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with  the  principals  involved  with  the  program  at  Headquarters 
Marine  Corps.  Second,  a  team  of  this  type  did  not  exist 
at  Headquarters  Karine  Corps  and  the  Development  Center's 
MAG IS  systems-engineering  team  conveniently  filled  this  void. 
Finally,  the  principal  managers  involved  v/ith  the  MAGIS 
program  at  Headquarters  Marine  Corps  were  very  receptive  to 
working  with  the  team  on  a  direct  and  personal  basis. 

The  MAGIS  systems-engineering  team,  was  to  be  initially 
concerned  v/ith  the  engineering  and  technical  aspects  of  system 
development  (Navy,  1972a).  Hov/ever,  the  scope  of  the  team's 
responsibilities  was  quickly  broadened  to  include  operational 
considerations,  recommended  revisions  to  Marine  Corps  Policy, 
and  support  of  the  overall  systems-acquisiticn  decision  process, 
including  the  Marine  Systems  Acquisition  Review  Council 
([M]SARC)  proceedings.  From  January,  1973,  until  August, 

1975,  the  MAGIS  systems-engineering  team  carried  ou.  many  of 
the  general  functions  and  methodology-associated  tasks  out¬ 
lined  in  Chapter  4  for  such  teams.  A  representative  set  of 
these  functions  and  tasks  are  described  below: 

1.  Establishment  of  the  Requirements  Baseline.  The 
MAGIS  systems-engineering  team  assisted  in  establishing  the 
requirements-def inition  baseline  and  operational  concepts  for 
the  system.  This  effort  was  carried  out  by  the  team's  working 
closely  with  the  several  contractor  analysis  groups  assigned  to 
the  program.  The  contractor  and  government  analysis  groups 
included  Texas  Instruments,  working  on  the  Imagery 
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Interpretation  Segment;  General  Electric,  on  the  Remote  Input/ 
Output  System  segment;  Systems  Development  Corporation,  on  the 
Intelligence  Analysis  Storage  and  Retrieval  Segment;  Bunker 
Ramo  Corporation  for  the  Signal  Intelligence  Electronic 
Warfare  coordination  Center;  Naval  Missile  Center,  Point 
Mugu,  California,  on  the  Tactical  Electronic  Reconnaissance 
Processing  and  Evaluation  Segment;  and  the  TIPI  Program 
Office,  working  on  the  overall  system.  The  many  varied 
products  of  these  analysis  groups  were  synthesized  by  the 
MAG IS  systems-engineering  team  and  conveyed  to  the  Development 
Center  managers  and  the  principals  at  HQMC  primarily  concerned 
with  the  program.  This  w as  done  by  holding  a  number  of 
working  sessions  with  the  Director  of  Intelligence  HQMC; 

Deputy  Director  Intelligence  HQMC;  Chief  of  Staff  of  the 
Development  Center;  and  the  MAGIS  systems-engineering  team. 
Throughout  these  working  sessions,  there  was  a  free  inter¬ 
change  of  ideas  and  concepts  related  to  operational  require¬ 
ments,  system-employment  procedures,  and  development  approaches. 

These  working  sessions  resulted  in  the  development  of 
a  document  entitled,  MAGIS  Development  and  Operational  Concepts 
(Navy,  1973b) .  It  included  operational  and  development  con¬ 
cepts  for  each  segment,  interactions  between  the  system  seg¬ 
ments,  a  system-depolyment  concept,  a  system-maintenance  con¬ 
cept,  a  system-communications  concept,  a  set  of  constraints 
on  system  development  and  employment,  and  a  plan  to  acquire 
each  segment.  Most  work  required  to  develop  this  document 
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was  carried  out  by  the  MAGIS  systems-engineering  team.  This 
effort  had  many  facets  that  are  analogous  to  the  analysis 
required  to  develop  what  is  now  called  the  Mission  Element  Meed 
Statement  (MENS)  for  systems-acquisition  programs.  Addition¬ 
ally,  much  of  the  analysis  effort  required  to  develop  the  MAGIS 
Development  and  Operational  Concepts  was  similar  to  that 
required  to  carry  out  Task  #1  of  the  systems-engineering 
methodology  outlined  in  Chapter  4  (p.  1S9)  . 

2.  Development  of  an  Alternative  Candidate  Solution 
for  the  Sy stems-Devel opment  Problem .  The  MAGIS  systems- 
engineering  team  developed  a  new  alternative  candidate  solution 
for  the  systems-development  problem.  This  solution  addressed 
the  problem  of  the  apparent  parallel  development  of  three 
of  the  MAGIS  segments — the  Remote  Input  Output  System  segment, 
the  Intelligence  Analysis  Storage  and  Retrieval  Segment,  and 
the  Signal  Intelligence  Electronic  Warfare  Coordination 
Center  segment.  A.s  noted  in  The  Period  in  Development  When 
the  Program  Suffered  from  the  Lack  of  a  Systems-Engineerina 
Approach  (p.  275) ,  these  three  separate  MAGIS  functional 
segments  were  being  developed  to  meet  the  same  intelligence¬ 
processing  requirements  at  various  deployment  levels. 

The  Intelligence  Analysis  Storage  and  Retrieval  (IA/SR) 
segment  was  to  be  deployed  at  the  Marine  Amphibious  Force 
echelon  for  the  atuomated  processing  of  combat  intelligence. 

The  Remote  Input  Output  System  segment  was  to  perform 

the  same  function  at  the  Marine  Air  Wing  and  Marine  Division 
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echelons.  Finally,  the  Signal  Intelligence/Electronic 
Warfare  Coordination  Center  (S/EWCC)  segment  was  to  provide 
automated  processing  of  special  intelligence  (i.e.,  electronics). 
This  development  approach  created  interoperability  problems 
between  the  various  segments  of  the  system.  It  also  increased 
both  development  and  procurement  costs  since  each  operational 
function  and  deployment  echelon  was  considered  separately. 

The  development,  by  the  MAG IS  systems-engineering  team, 
of  a  new  alternative  candidate  solution  for  the  system 
depended  on  the  analysis  of  the  operational  requirements 
discussed  above.  This  analysis  led  to  several  significant 
conclusions : 

1.  That  the  three  segments  previously  mentioned 
(IA/SR,  RIOS,  and  S/EWCC)  differed  significantly  only  in 
the  types  and  volumes  of  data  processed  by  each  segment  and 
the  particular  command  level  at  which  the  segments  operated. 

2.  That  the  differences  in  data  were  related  to 
security  considerations  for  handling  special  intelligence 
data  in  the  collection  and  direction  phases  of  the  classical 
intelligence  cycle  (see  Figure  32). 

3.  That  the  projected  technical  capabilities  (hard¬ 
ware,  software,  and  means  of  communications)  for  each  segment 
were  basically  the  same. 

4.  That  except  for  the  security  requirements  to 
separate  special  intelligence  data  from  other  types  of 
intelligence  data,  it  was  possible  to  design  a  single  segment 
that  would  meet  all  three  operational  requirements. 
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Based  on  the  above  conclusions,  the  MAGIS  systems- 
engineering  team  developed  the  Modularity  Concept  as  an  alter¬ 
native  candidate  solution  to  resolve  the  problem  of  the 
parallel-segment  developments  (Navy,  1973e) .  One  part  of  the 
modularity  concept  is  stated  as  follows: 

That  a  single  processing  capability  be  developed  based  on 
a  systems  approach  which  will  have  the  ability  to: 

a.  Handle  several  levels  of  throughput; 

b.  Address  a  variety  of  functions; 

c.  Support  multiple  canmand  echelons. 

This  alternative  candidate  solution  was  to  use  the 
Intelligence  Analysis/Storage  and  Retrieval  (IA/SR)  segment 
as  the  design  baseline  for  a  single  processing  capability 
to  replace  the  previously  mentioned  three  segments.  It 
was  to  be  housed  in  two  distinct  shelters  (8'X  8 'X  20')  : 

One  shelter  was  designed  as  an  automatic  data-processing/ 
communications  capability  and  the  other  was  designed  to  house 
four  analysts  to  process  the  all-source  intelligence  data. 
Depending  on  the  throughput  requirements  of  the  system, 
additional  analyst  shelters  could  be  added  to  the  system, 
hence  the  term  modular  concept.  This  new  single-processing 
capability  was  to  be  called  the  Intelligence  Analysis  Center 
( I AC) .  Figure  37  depicts  the  combining  of  the  three 
intelligence-processing  segments  existing  in  1972  into  the 
single  IAC  capability. 

The  development  of  a  new  alternative  candidate 
solution  by  the  MAGIS  systems-engineering  team  is  an  example 
of  a  systems-engineering  team's  performing  its  general  function 


1972 


1974 


Figure  37.  Implementation  of  the  Modularity  Concept  with 

the  Intelligence  Analysis  Center  (IAC) .  This  graphic 
portrays  the  translation  of  the  three  previous  MAGIS 
segments  into  a  single  intelligence  processing  capability 
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to  carry  out  critical  analysis  procedures  that  have  mayor 
policymaking  or  decision-making  significance  to  the  organiza¬ 
tion  (see  Chapter  4,  p.246).  Although  this  alternative  candi¬ 
date  solution  was  developed  in  1973  by  the  MAGIS  systems- 
engineering  team,  the  idea  had  to  be  approved  by  top-level 
decision  makers  before  it  could  be  implemented.  The  team's 
effort  to  secure  this  approval  is  the  next  example  of  the 
team's  employment. 

3.  Evaluation  of  the  New  Alternative  Candidate 
Solution  and  Communications  with  Policymakers  and  Decision 
Makers.  Once  the  modularity  concept  of  a  single  intelligence¬ 
processing  capability  (the  Intelligence  Analysis  Center)  was 
developed  by  the  MAGIS  systems  engineering  team,  it  had  to 
be  evaluated  for  possible  implementation.  This  analysis  was 
carried  out  jointly  by  the  MAGIS  systems-engineering  team 
and  a  General  Electric  analysis  group  attached  to  the  A.ir 
Force  TIPI  Program  Office.  Different  evaluation  factors  were 
considered  in  relation  to  the  IAC  alternative  candidate  solu¬ 
tion,  including  RDT&E  and  procurement  costs,  maintenance  and 
training  requirements,  personnel  requirements,  management 
effort,  policies  and  procedures  relating  to  the  handling  of 
special  intelligence,  system  supportability  and  inter¬ 
operability.  The  results  of  these  analysis  efforts  are 
presented  in  Remote  Input  Output  System  (RIOS)  Design  Trade¬ 
off  Analysis  Report ,  Systems  Integration  and  Checkout  Contract , 
for  the  TIPI  System  (GE,  1973);  Utilization  of  Modular 
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Equipment  within  the  MAGIS  Development  Program,  (Navy,  1973e) , 
and  Budgetary  Estimates  of  the  Marine  Air /Ground  Intelligence 
System  (MAGIS)  Equipment  Development  and  Procurement  (Navy, 
1973a)  . 

The  data  developed  during  the  197  3  analysis  may  be 
conviently  presented  using  a  technique  documented  in 
De  Neufville  and  Stafford  (1971).  This  technique  is  called  an 
impact-incidence  matrix  and  is  designed  to  inform  decision 
makers  about  the  distribution  of  the  benefits  and  costs  of 
an  alternative.  The  analyst  can  summarize  the  several 
measures  of  effectiveness  associated  v/ith  a  particular  alter¬ 
native  on  a  single  table,  such  as  that  shown  in  Table  11. 

By  using  the  same  format  for  all  competing  alternatives, 
comparisons  are  facilitated.  A  summarization  of  the  data 
developed  as  a  result  of  the  1973  analysis  for  the  IAC  alter¬ 
native  candidate  solution  is  presented  in  Table  12.  In 
pursuit  of  the  goal  of  brevity  of  this  dissertation  a  more 
detailed  analysis  is  not  presented. 

Subsequent  to  the  evaluation  of  the  IAC  alternative, 
it  had  to  be  presented  to  top-level  policymakers  and  decision 
makers  for  review  and  possible  approval.  This  was  done  through 
several  working  conferences  between  Development  Center  and 
Headquarters  Marine  Corps  personnel.  Initially,  a  set  of 
conferences  were  held  with  the  Director  of  Intelligence  and 
his  staff  at  Headquarters  Marine  Corps  to  brief  the  Director 
of  Intelligence  on  the  operational  policy  implications  and 
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Groups  Impacted 


Impacts 

Directly  Indirectly  Estimated 

Estimated  Estimated  Numerically 

$  $  not  in  $ 

ai  a2  S3  "*■  kj  ^2  "**  C1  C2 


Estimated 
Qualitatively 
in  words 


d  d 
1  2 


Indirectly  B  j 


Special  Cj 

interests  C2 


SOURCE:  de  Neufville  and  Stafford  (p.  249,  1971) 

NOTES:  The  vertical  axis  represents  the  groups  impacted  by  a  particular 
alternative;  the  horizontal  axis  represents  measures  of  effectiveness  by 
which  the  costs  and  benefits  of  a  particular  alternative  are  evaluated. 
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employment  opportunities  that  would  result  from  implementing 
a  single  intelligence-processing  capability,  or  IAC .  The 
technical  ramifications  of  implementing  the  IAC  development 
approach  were  presented  by  members  of  the  systems-engineering 
team.  Detailed  discussions  were  held  on  the  need  to  revise 
the  Marine  Corps'  operational  policy  relating  to  the  security 
requirements  for  handling  special  intelligence  if  the  IAC 
development  approach  was  to  be  implemented.  As  a  direct 
result  of  these  discussions,  official  approval  was  given  to 
revise  Marine  Corps'  operational  policies  for  the  handling 
of  special  intelligence  thus  paving  the  way  for  the  implementa¬ 
tion  of  the  IAC  development  approach  (Navy,  1973d) . 

Discussions  were  also  held  with  the  Deputy  Chief  of 
Staff,  Research  and  Development,  Headquarters  Marine  Corps 
and  representatives  from  other  staffs  at  Headquarters 
Marine  Corps.  The  MAGIS  systems-engineering  team  members 
presented  material  on  different  topics  that  had  been  analyzed 
by  the  MAGIS  team,  including  the  IAC  development  approach. 
Again  as  a  direct  result  of  discussions,  Headquarters  Marine 
Corps'  official  approval  was  given  to  begin  designing  the 
IAC  (Navy,  1973c) .  Figure  37  is  a  graphical  representation 
of  the  three  previous  MAGIS  segments  being  translated  into  a 
single  intelligence-processing  capability,  the  IA.C .  Through¬ 
out  the  conferences  discussed  above,  the  MAGIS  systems- 
engineering  team  performed  its  general  function  to  translate 
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the  results  of  internal  analy si s  efforts  into  management 
information  (see  Chapter  4,  p.  246). 

4.  Communication  of  Policy  Guidance  From  Headquarters 
to  the  Analysis  Groups  Working  on  the  Program.  When  the 
Headquarters  Marine  Corps'  policy  guideline  (Navy,  1973c,  1973d) 
was  issued  for  the  IAC  development  approach,  the  MAGIS  systems- 
engineering  team  had  to  convey  this  policy  guidance  to  the 
analysis  groups  working  on  the  program.  The  members  of  the 
team  worked  closely  with  representatives  of  the  TIFI  Program 
Office  to  insure  that  the  Air  Force  Development  Agency  for 
MAGIS  understood  the  newly  defined  Marine  Corps'  operational 
policy.  Detailed  working  sessions  were  held  with  the  separate 
TIPI  Program  Office  sections  to  properly  identify  the  many 
ramifications  of  implementing  the  new  policies  for  handling 
special  intelligence. 

The  MAGIS  systems-engineering  team  also  worked  closely 
with  the  intital  design  contractor  for  the  IAC  segment,  the 
General  Electric  Company.  The  team  members  had  to  establish 
the  new  requirements  for  handling  special  intelligence 
and  the  single  intelligence-processing  capability  in  much 
greater  detail  than  in  the  two  Marine  Corps  policy  and 
decision  letters  for  the  IAC  segment.  The  end  product  of 
this  effort  was  the  Summary  Technical  Report  for  the  Intelli¬ 
gence  Analysis  Center  (IAC)  Segment  for  the  MAGIS ,  (GE,  1974a). 
This  document  detailed  the  compression  of  the  three  previous 
MAGIS  segments  into  the  single  intelligence-processing 
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capability  of  the  IAC  and  included  relevant  design  information 
on  the  special-intelligence-handling  requirement.  Table  13 
lists  the  MAGIS  development  segments  in  1S74  after  the 
initial  adoption  of  the  IAC  as  a  single  intelligence¬ 
processing  capability. 

Throughout  the  working  sessions  discussed  above,  the 
MAGIS  systems-engineering  team  performed  its  general  function 
to  keep  the  analysis  groups  informed  of  policy  revisions  that 
could  impact  analysis  procedures  (see  Chapter  4,  p.246). 

5.  Development  of  Evaluation  Areas  and  Factors. 

Once  the  Intelligence  Analysis/Storage  and  Retrieval  (IA./SR) 
design  was  approved  as  the  IAC  segment  baseline,  the  Marine 
Corps  would  have  to  consider  a  full-scale  development  phase 
for  the  IAC.  During  the  full-scale  development  phase,  a  test 
model  IA.C  would  be  designed  and  built  for  development  test 
(DT)  II  and  initial  operational  test  and  evaluation  ( IOT&E) . 
Before  going  to  full-scale  development,  the  Marine  Corps  had 
to  consider  major-decision  Milestone  II.  Major-decision 

4 

Milestone  I  was  waived;  the  strong  design  baseline  that  had 
evolved  out  of  the  previous  IA/SR  segment  development  was 

4 

The  Intelligence  Analysis/Storage  and  Retrieval  (IA/SR)  segment, 
the  IAC  design  baseline,  was  then  (1974)  in  the  full-scale  development  phase. 
This  design  baseline  represented  an  alternative  candidate  solution  that  had 
been  approved  at  major-decision  Milestones  I  and  II  for  the  IA/SR  program 
and  had  successfully  completed  the  conceptual  and  demonstration-and- 
validation  phases.  Therefore,  the  IAC  systems-acquisition  program  did 
not  have  to  be  subjected  to  major-decision  Milestone  I.  This  had  been 
effectively  accomplished  in  the  IA/SR  program. 
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regarded  as  the  demonstration  and  validation  phase  for  the  IAC 
program. 

In  preparing  for  the  major-decision  Milestone  II, 
the  MAGIS  systems-engineering  team  worked  with  Development 
Center  Management  and  Headquarters  Marine  Corps  personnel 
developing  overall  evaluation  areas  and  factors  to  be  consid¬ 
ered.  Development  and  production  costs,  development  risk, 
supportability  of  the  system  in  the  field,  technical 
performance  characteristics,  and  overall  development 
schedules  were  all  evaluated.  Each  area  was  thoroughly 
analyzed  to  determine  specific  evaluation  factors  to  be  used 
in  the  major  milestone  II  decision  process.  Factors  similar 
to  those  persented  in  Table  9  (Evaluation  Treas  and  Factors, 
p.209)  were  developed.  During  this  same  time,  the  MAGIS 
systems-engineering  team  updated  the  constraints  and  goals  for 
the  IAC  development  program.  These  constraints  and  goals 
were  documented  in  a  letter  from  the  Commanding  General  of 
MCDEC  to  the  Program  Manager  of  the  TIPI  Program  Office  (Navy, 
1974b)  . 

In  doing  the  analyses,  the  MAGIS  systems-engineering 
team  performed  several  functions  analogous  to  those  required 
to  implement  Task  #3  (p.  218  )  for  the  systems-engineering 
methodology  developed  in  Chapter  4. 

6.  Assistance  in  Defining  the  Major  Milestone  II 
Decision  Problem  and  in  Evaluating  Alternative  Candidate 
Solutions .  In  preparing  for  the  major-decision  Milestone  II 
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(or  Marine  [M]  SA.P.C  II);  the  MAGIS  systems-engineering  team 
worked  closely  with  Headquarters  Marine  Corps'  personnel  tc 
define  the  decision  problem.  The  key  issue  in  the  acquisition 
program  at  this  time  was  which  development  approach  (alter¬ 
native  candidate  solution)  should  be  used  for  the  full-scale 
development  phase  of  the  Intelligence  .Analysis  Center  (IA.C)  . 
This  issue  related  primarily  to  the  equipment  to  be  used  in 
developing  the  IAC. 

The  baseline  design  for  the  IAC,  the  Intelligence 
Analysis/Storage  and  Petrieval  (IA/SP)  segment,  had  been 
developed  using  minicomputers,  the  AN/UYK-12.  The  Air  Force 
TIPI  Program  Office  supported  the  development  of  the  IAC  using 
these  minicomputers.  On  the  other  hand,  the  Marine  Corps  had 
recently  directed  that  the  AN/UYK-20,  the  Navy-standard 
minicomputer,  be  used  within  the  Marine  Corps.  The  Navy 
had  also  developed  peripheral  equipment  adapted  to  the  Navy- 
standard  AN/UYK-20  minicomputer.  Several  factions  within 
the  Marine  Corps  supported  the  development  of  the  IAJ3  using 
the  AN/UYK-20  and  its  peripheral  equipment.  The  IAC  was 
a  unique  Marine  Corps  development  program  and  the  Marine 
Corps  was  funding  the  PDTSE  and  production  costs.  Therefore, 
the  issue  of  which  equipment  package  to  use  was  resolved  within 
the  Marine  Corps  and  the  Air  Force  was  requested  tc  implement 
the  decision. 

Prior  to  the  (M)  SAP.C  II  decision  point,  a  number  of 
analysis  groups  considered  what  suite  of  equipment  should 


be  used  in  the  IA.C  acquisition  program  for  the  full-scale 
development  phase.  General  Electric  Company,  the  Systems 
Development  Corporation ,  the  TIPI  Program  Office,  and  the 
Naval  Surface  Weapons  Center,  Dahlgren,  Virginia,  all  analyzed 
the  problem.  The  MAGIS  systems-engineering  team  reviewed  the 
results  of  these  analysis  efforts,  synthesized  their  results, 
and  developed  a  background  document  on  the  problem.  The 
Deputy  Chief  of  Staff,  Research  and  Development,  Headquarters 
Marine  Corps  was  then  briefed  in  August,  1974.  At  that 
time,  he  directed  that  the  matter  be  referred  to  a  Headquarters 
Marine  Corps  In-Progress  Review  Committee,  which  consisted 
of  several  other  general  officers.  Before  presentation  to  the 
review  committee,  the  MAGIS  systems-engineering  team  was 
directed  to  develop  the  life-cycle  costs  (LCC)  for  the  system 
with  both  alternatives — of  developing  the  IAC  with  the 
A1J/UYK-2 0  or  the  AN/UYK-12  minicomputers  (Navy,  1974d)  . 

The  MAGIS  systems-engineering  team  performed  the 
I.AC-segment  life-cycle-cost  (LCC)  analysis  during  August  and 
September  of  19 7 4  with  the  aid  of  a  FORTRAN  computer  model 
originally  developed  by  the  General  Electric  Company  (GE,  1974b) 
but  revised  by  the  MAGIS  systems-engineering  team.  The 
analysis  arrived  at  a  cost  for  the  FDT&E,  production,  and 
operations  for  the  two  separate  alternatives.  These  figures 
were  forwarded  to  Headquarters  Marine  Corps  from  the  Develop¬ 
ment  Center  (Navy,  197dd) . 
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On  October  3,  1974,  the  Headquarters  Marine  Corps' 
In-Progress  Review  Committee  met  to  consider  the  IAC  systeros- 
acquisition  program.  This  committee  meeting  was  a  precursor  to 
the  (M) SARC  II  decision  point.  The  Committee  considered 
three  principal  factors  in  deciding  which  alternative  candi¬ 
date  solution  should  be  used  in  the  IAC  full-scale  develop¬ 
ment  phase  (Navy,  1974a) :  the  Marine  Corps  standardization 
program  for  minicomputers,  life-cycle  costs  of  the  systems, 
and  the  logistic  and  maintenance  supportability  of  the  system 
in  the  field.  The  last  two  factors  had  been  subjected  to 
extensive  analysis  by  the  MAGIS  systems-engineering  teams. 

The  In-Progress  Review  Committee  was  briefed  on  four  alterna¬ 
tive  candidate  solutions  for  the  IAC  development  program. 
Different  MAGIS  team  members  addressed  various  aspects  of 
the  decision  process.  The  life-cycle  cost  analysis,  in 
particular,  became  a  critical  factor  in  the  decision  process. 
Ultimately,  the  committee  decided  to  make  a  formal 
recommendation  to  the  (M) SARC  II  panel  that  the  IAC  be  devel¬ 
oped  using  the  AN/UYK-20  minicomputer  and  its  associated 
peripheral  equipment  (Navy,  1974a) . 

In  performing  the  above  services,  the  MAGIS  systems- 
engineering  team  did  much  of  the  analysis  required  to 
support  major-decision  Milestone  II  as  projected  by  Task  #3 
(p.  218)  of  the  systems-engineering  methodology  outlined  in 
Chapter  4.  The  MAGIS  team  also  carried  out  the  following 
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general  systens-engineering  team  functions  as  outlined  in 
Chapter  4 : 

a.  Translate  the  results  of  internal  analysis 
efforts  into  management  information; 

b.  interpret  the  results  of  other  organizations ' 
analyses  for  management; 

c.  carry  out  critical  analysis  procedures  that 
have  major  policymaking  or  decision  making  significance  to 
the  organization. 

7.  Assistance  in  the  Preparations  for  the  Marine 
(M)SARC  II  Presentations  .  Much  of  the  analysis  described  in 
the  previous  example  was  revised  and  updated  by  the  MAG IS 
systems-engineer ing  team  to  support  the  (M)  5  ARC  II  proceedings 
for  the  I AC  segment.  Prior  to  the  meeting,  members  of  the 
MAG IS  team  attended  conferences  and  working  sessions  to 
prepare  Headquarters  Marine  Corps  personnel  for  the  (M) SARC 
II  decision  milestone.  The  general  systens-engineering  team's 
tasks  and  functions,  as  described  above  under  6.,  were 
carried  out  by  the  MAGIS  team  to  prepare  top-level  management 
for  major-decision  Milestone  II.  These  efforts  culminated 
in  the  decision  by  the  (M) SARC  panel  of  general  officers  to 
implement  the  IAC  full-scale  development  phase  using  the 
AN/UYK-2Q  minicomputers  (Navy,  19 7 5a, b) . 

The  following  benefits  were  credited  to  the  employment 
of  the  MAGIS  systens-engineering  teams  from  January,  1973, 
to  August,  1975: 
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1.  Two  systems-acquisition  programs  were  completely 
deleted  from  the  Marine  Corps'  operational  requirements  when 
the  three  previous  ['LAG IS  segments  were  combined  into  the 
Intelligence  Analysis  Center  (IAC)  segment. 

2.  Savings  conservatively  estimated  at  $9 . 0  million 
for  RDT&E  and  $12.2  million  for  production  were  realized 
when  adoption  of  the  modularity  concept,  developed  by  the 
MAG IS  team  and  adopted  by  the  Marine  Corps,  caused  a  conser¬ 
vatively  estimated  reduction  in  the  RDT&E  and  production 
budgets  of  $9.0  million  and  $12.2  million.  The  impact  of 
these  reductions  may  be  realized  by  comparing  the  Marine 
Corps'  total  RDT&E  budget  of  approximately  $30-40  million 

a  year  for  that  period  to  the  total  minimum  savings  of 
$21.2  million. 

3.  The  Marine  Corps  potential  interoperability  pro¬ 
blems  because  of  the  three  different  segments  existing  in 
1972  were  eliminated  by  establishing  the  single  intelligence¬ 
processing  capability,  the  IAC. 

The  benefits  above  resulted  from  the  employment  of 
the  MAGIS  systems-engineering  team  and,  particularly,  the 
role  the  team  played  in  the  decision  process  as  outlined  in 
the  seven  example  activities  described  in  this  section.  A 
graphic  summary  of  the  activities  of  the  MAGIS  systems-.  . 
engineering  team  from  February,  1973,  to  August,  1975,  is 
presented  in  Figure  38.  This  figure  portrays  the  team  in 
the  role  that  it  performed  to  bridge  the  gap  between 
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policymakers  and  decision  makers  and  the  various  analysis 
and  engineering  groups  working  on  the  program. 

Summary  and  Conclusions 

This  chapter  addressed  a  real-world  Defense  systems- 
acquisition  program  and  its  associated  decision  process.  £ 
joint  development  effort  between  the  Marine  Corps  and  the 
Air  Force  to  develop  a  computer-based  system  for  processing 
tactical  intelligence  was  studied  from  a  historical  prospec¬ 
tive.  Specifically,  this  program  was  examined  to  determine 
how  the  employment  of  a  systems-engineering  team  and  approach 
impacted  the  system-development  program  and  associated 
decision  process. 

First  the  chapter  provided  sufficient  historical 
and  background  information  on  the  program  to  establish  a 
framev;ork  within  which  the  specific  systems  problems  could 
be  studied.  Second  the  period  of  the  development  when  the 
program  suffered  from  the  lack  of  a  systems-engineering  approach 
was  studied.  Some  problems  from  this  period  of  the  develop¬ 
ment  effort  were  redundant  development  efforts,  excessive 
costs,  ill-conceived  operational  concepts,  and  a  lack  of 
insight  into  the  systems  problems  by  top-level  management 
within  the  Marine  Corps.  This  chapter  also  covered  the  period 
of  development  when  the  program  benefited  from  a  systems- 
engineering  approach.  It  was  during  this  period  that  a 
systems-engineering  team  was  formed  and  used  to  support 


the  decision  process  as  the  systems-development  program 
approached  the  major  Milestone  II  decision  point.  Through¬ 
out  this  period,  the  systems-engineering  team  worked  closely 
with  both  the  Marine  Corps'  decision  makers  and  the  various 
analysis  and  engineering  groups  working  on  the  program. 
Therefore  based  on  the  material  developed  in  this  chapter, 
the  following  conclusions  are  drawn: 

1.  That  the  MAGIS  program  is  representative  of  a 
Defense  systems-acquisition  program  to  which  a  systems- 
engineering  approach  could  be  applied. 

2.  That  a  systems-engineering  team,  possessing  the 
general  team  characteristics  as  described  in  Chapter  4,  was 
formed  to  assist  in  the  decision  process  associated  with  the 
MAGIS  development  program. 

3.  That  the  systems-engineering  team  that  was  formed 
carried  out  several  of  the  general  functions  and  the 
methodology-associated  tasks  outlined  in  Chapter  4  for  such 
teams.  The  following  functions  and  tasks  were  performed  by 
the  team: 

a.  Assisted  in  establishing  the  requirements- 
definition  baseline  and  operational  concepts  for  the  system. 

b.  Developed  a  new  alternative  candidate  solution 
for  the  systems-development  problem  based  on  a  clearly  defined 
set  of  operational  requirements. 

c.  Assisted  in  evaluating  the  new  alternative 
candidate  solution  (technical  approach)  and  communicated  the 
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nature  of  this  new  alternative  to  policymakers  and  decision 
makers.  The  team  also  assisted  them  in  understanding  the 
operational  policy  implications  and  employment  opportunities 
that  would  be  associated  with  implementing  the  new  technical 
approach.  These  efforts  resulted  in  a  new  operational  policy 
for  the  employment  of  the  system  in  the  field. 

d.  .Assisted  in  communicating  the  Headquarters 
Marine  Corps'  operational  policy  guidance,  which  would  impact 
systems  development,  to  the  various  analysis  and  engineering 
groups  working  on  the  program. 

e.  Assisted  in  the  development  of  evaluation 
areas  and  factors  that  would  enable  discrimination  between 
the  alternative  candidate  solutions  during  the  major- 
decision  Milestone  II  process. 

f.  Assisted  the  Marine  Corps'  decision  makers  in 
defining  the  decision  problem  and  in  evaluating  and  selecting 
the  alternative  candidate  solutions  at  the  Milestone  II 
decision  point. 

g.  Assisted  in  the  preparations  for  the  actual 
(M) SARC  presentations  for  the  Milestone  II  decision 
point. 

4.  That  based  on  the  comments  outlined  above  in  3., 
the  systems-engineering  team  associated  with  the  MAG IS  develop¬ 
ment  program  acted  to  bridge  the  gap  between  policy  makers  and 
decision  makers  and  the  various  analysis  and  engineering 
groups  working  on  the  program. 
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5.  That  the  systems-engineering  team  substantially 
contributed  to  the  improvement  of  the  MAGIS  development 
program  by: 

a.  Assisting  in  the  development  and  adoption  of 
a  more  viable  set  of  operational  employment  concepts  for  the 
system. 

b.  Assisting  in  the  redirection  of  the  overall 
systems-development  program,  which  resulted  in  significant 
cost  savings  during  the  full-scale  development  phase  of  the 
svstems-acquisition  process. 

c.  Assisting  decision  makers  and  policymakers 
in  more  effectively  handling  their  responsibilities. 

d.  Assisting  engineering  and  analysis  groups 
working  on  the  program  to  better  understand  policy  guidelines 
from  the  Headquarters  Marine  Corps  level. 
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Chapter  6 


SUMMARY,  CONCLUSIONS,  CONTRIBUTIONS,  AND 
RECOMMENDATIONS  FOR  FUTURE  RESEARCH 

Summary 

This  research  effort  investigated  the  management  of 
the  systems-acquisition  process  within  the • Department  of 
Defense  (DOD) .  It  indicated  how  the  DOD  could  apply  systems- 
engineering  techniques  to  achieve  a  more  effective  approach 
to  decisions  involved  in  the  development  and  procurement  of 
Defense  systems. 

Initially,  a  review  was  presented  of  the  evolution  of 
DOD  acquisition  policies  and  procedures  over  the  last  several 
years.  The  existing  policy  structure  for  the  systems- 
acquisition  process  was  then  analyzed  with  emphasis  on  the 
phases,  activities,  and  decisions  of  the  process.  The  phases 
and  activities  of  the  systems-acquisition  process  were  then 
compared,  respectively,  to  the  various  phases  and  steps  of 
the  systems-engineering  framework.  There  was  a  respective 
pairwise  relationship  between  the  phases,  activities,  and 
discipline  requirements  of  the  systems-acquisition  process 
and  the  phases,  steps,  and  knowledge  requirements  of  the 
systems-engineering  framework. 

Using  the  systems-engineering  framework,  the  nature 
and  structure  of  the  Defense  systems-acquisition  decision 
process  was  examined  to  identify  analysis  areas  where  the 


320 


application  of  a  systems-engineering  methodology  would  be 
most  beneficial.  Selection  criteria  and  organizational  con¬ 
siderations  were  identified  for  choosing  appropriate  systems- 
engineering  tools  and  techniques  for  use  in  the  methodology. 

A  methodology  with  several  specific  analysis  tasks  was  then 
developed.  The  analysis  procedures  associated  with  these 
tasks  were  dependent  on  the  particular  stage  of  the  acquisi¬ 
tion  process  and  the  nature  of  the  decision  under  considera¬ 
tion.  Implementation  of  the  methodology  was  discussed  next. 

In  this  portion  of  the  effort,  particular  emphasis  was  placed 
on  the  formation  and  proper  utilization  of  systems-engineering 
teams  to  support  policymakers  and  decision  makers  in  carrying 
out  their  responsibilities.  The  functions  and  characteris¬ 
tics  nominally  associated  with  these  systems-engineering 
teams  were  presented. 

Finally,  a  historical  analysis  was  presented  of  a 
Defense  systems-acquisition  program.  The  program  analyzed 
was  oriented  toward  the  development  of  a  tactical  intelli¬ 
gence  system.  The  principal  thrust  of  this  analysis  was  to 
demonstrate  the  viability  and  usefulness  of  employing  both  a 
systems-engineering  approach  and  team  in  supporting  the 
Defense  systems-acquisition  decision  process. 

Cone lusions 

Based  on  the  material  developed  in  this  systems  study, 
several  conclusions  may  be  drawn.  Four  major  conclusions 
are  presented  here  in  priority  order. 
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1.  The  systems-acquisition  process  may  be  effectively 
modelled  by  a  systems-engineering  framework.  The  two  are 
similar  in  that  there  is  a  respective  pairwise  correlation 
between  the  phases  and  steps  of  the  framework  and  the  phases 
and  activities  of  the  process.  Additionally,  both  the 
systems-acquisition  process  and  systems  engineering  are 
multidisciplinary  activities. 

2.  The  systems-acquisition  process  could  benefit  from 
the  application  of  a  systems-engineering  methodology  to  support 

(1)  the  problem  definition  in  the  mission-analysis  phase  and 

(2)  the  major-decision  milestones  of  the  process.  The 
analysis  techniques  selected  as  procedures  in  the  methodology 
should  be  dependent  on  both  organizational  considerations 
and  the  stage  of  the  development  program  under  evaluation. 

3.  Effective  implementation  of  the  methodology 

is  critically  dependent  on  the  formation  and  proper  use  of 
systems-engineering  teams  working  with  top-level  managers 
within  the  Office  of  the  Secretary  of  Defense  and  the  several 
service  headquarters.  The  principal  role  for  these  teams  is 
that  of  bridging  the  gap  between  policymakers  and  decision 
makers  and  the  various  analysis  and  engineering  groups 
working  on  the  program.  In  this  role,  they  can  significantly 
enhance  communications  for  the  development  programs  and 
thereby  improve  the  overall  management  of  the  systems- 
acquisition  process.  These  improvements  would  be  reductions 
in  research,  development,  test  and  evaluation,  and 


production  costs  and  in  development  time;  they  would  enhance 
supportability  and  interoperability  once  the  system  is  deployed. 

4.  The  systems-engineering  team  associated  with  the 
Marine  Air/Ground  Intelligence  System  (MAGIS)  development 
program  (selected  for  historical  analysis)  bridged  the  gap 
between  Marine  Corps  Headquarters  (policy  and  decision  makers) 
and  the  several  analysis  groups  working  on  the  program.  The 
employment  of  the  systems-engineering  team  in  this  role  pro¬ 
vided  substantial  benefits  to  the  Marine  Corps  from  1972  to 
1975  and  is  a  good  example  of  the  successful  application  of 
the  methodology,  which  is  the  subject  of  this  dissertation. 

Contributions 

The  contributions  of  the  research  to  the  systems- 
management  component  of  systems  engineering  are  identified 
as  follows: 

1.  It  establishes  that  the  systems  acquisition  process 
may  be  appropriately  modelled  by  the  systems-engineering  frame¬ 
work  and  demonstrates  the  applicability  of  the  systems-engineering 
tools  and  techniques  to  the  activities  of  the  process. 

2.  It  demonstrates  how  the  systems-acquisition  deci¬ 
sion  process  may  be  supported  by  the  application  of  the 
systems-engineering-methodology  tasks  to  the  major-decision 
milestones . 

3.  It  provides  systems-acquisition  decision  makers 
and  policy  makers  a  means  by  which  they  can  enhance  their 
managerial  abilities — the  formation  of  the  systems-engineering 
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team,  as  discussed,  and  the  use  of  these  teams  in  the  roles 
described  for  them. 

4.  It  demonstrates  the  usefulness  of  the  systems- 
engineering  approach  and  the  role  of  systems-engineering 
teams  in  the  Defense  systems-acquisition  process  by  con¬ 
sidering  an  applied  problem. 

5.  It  establishes  a  foundation  for  the  future  study  of 
the  systems-acquisition  process  throughout  the  federal 
government. 

Recommendations  for  Future  Research 

Several  recommendations  for  future  research  suggest 
themselves : 

1.  Apply  the  systems-engineering  methodology  and 
associated  analysis  procedures  developed  in  the  research  to 
a  variety  of  active  Defense  systems-acquisition  programs. 

2.  Extend  the  systems-engineering -methodology  tasks 
to  other  activities  of  the  DOD  systems-acquisition  process. 

3.  Establish  criteria  to  determine  the  applicability 
of  a  systems-engineering  methodology  to  the  systems-acquisition 
process  within  other  federal  agencies. 

4.  Develop  a  specific  systems-engineering  methodology 
to  support  the  systems-acquisition  decision  process  in  other 
federal  agencies,  such  as,  the  Department  of  Transportation 

or  the  Department  of  Energy. 
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5.  Develop  a  global  systems-engineering  methodology 
to  support  the  systems-acquisition  decision  process  for  all 
federal  agencies  from  the  perspective  of  the  Office  of 
Federal  Procurement  Policy  within  the  Office  of  Management 
and  Budget. 

6.  Study  the  areas  of  interaction  between  the  federal 
government  and  the  industrial  sector  in  relation  to  the 
application  of  a  systems-engineering  methodology  to  the 
systems-acquisition  decision  process. 
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LIST  OF  TECHNICAL  TERMS 


The  following  is  a  list  of  definitions  selected  from  many  sources 
in  the  course  of  writing  this  dissertation  and  over  several  years  of  work¬ 
ing  in  the  field.  They  are  included  as  an  aid  to  the  reader  and  in  the 
interest  of  better  communications. 


Acquisition  program,  a  directed  effort  funded  either  throuah  procurement 

appropriations,  or  research,  development,  test,  and  evaluation  appro¬ 
priations  with  the  goal  of  providing  a  new  or  improved  capability 
in  response  to  a  validated  need.  An  acquisition  program  may  in¬ 
clude  either  development  or  procurement  of  systems,  subsystems, 
equipment,  munitions,  or  modifications  to  them,  as  well  as 
supporting  equipment,  systems,  projects,  and  studies. 

Acquisition  risk.  The  chance  that  some  element  of  an  acquisition  program 
produces  an  unintended  result  with  an  adverse  effect  on  system 
effectiveness  (technical  risk) ,  cost  (cost  risk) ,  or  availability 
for  deployment  (schedule  risk) .  Risk  could  also  be  considered 
in  relation  to  social,  environmental  and  political  factors. 

Acquisition  strategy.  A  general  set  of  concepts  for  handling  technical, 

business,  and  programmatic  matters  pertaining  to  the  management  of 
a  major  system.  Plans  provide  the  detailed  methods  for  the 
fulfillment  of  the  strategy. 

Agency  component.  This  is  a  major  organizational  subdivision  of  an  agency. 
For  example:  the  Army,  Navy,  Air  Force,  and  Defense  Supply  Agency 
are  agency  components  of  the  Department  of  Defense. 

Agency  missions.  Those  responsibilities  for  meeting  national  needs  which 
are  assigned  to  a  specific  agency. 

Availability .  a  measure  of  the  degree  to  which  an  item  is  in  the  operable 
and  commitable  state  at  the  start  of  a  mission  when  the  mission 
is  called  for  at  an  unknown  (random)  time. 

Budgeting .  For  the  purpose  of  planning,  programming  and  budgeting  system 
(PPBS) :  the  process  of  determining  short-range  detailed  allocation 
of  resources  to  execute  assigned  missions. 

Compatibility .  The  capability  of  two  or  more  operational  items/ systems 
to  exist  or  function  as  elements  of  a  larger  operational  system 
or  operational  environment  without  mutual  interference. 
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Conceptual  phase.  The  period  of  the  acquisition  process  which  deals 
with  the  identification  and  exploration  of  alternative  solu¬ 
tions  or  solution  concepts  to  satisfy  a  validated  need,  ususally 
through  the  use  of  contracts  with  competent  industrial  and 
educational  institutions.  This  phase  requires  the  active 
involvement  of  all  participating  commands  to  identify  the 
candidate  solutions  and  their  characteristics.  One  or  more 
of  the  selected  candidate  solutions  are  then  approved  for  entry 
into  the  Demonstration  and  Validation  phase. 

Constraint,  a  resource  limitation  which  may  be  specific  (e.g.,  the 

supply  of  skilled  manpower  of  a  particular  type  of  a  particular 
metal),  or  general  (e.g.,  total  available  funds). 

Cost  analysis.  A  process  employed  to  develop  or  assess  the  reasonableness 

and  validity  of  resource  requirement  estimates  for  military  systems 
and  programs.  This  process  includes  a  statement  or  report  of  the 
assessment  together  with  related  conclusions. 

Cost-effectiveness  analysis.  The  quantitative  examination  of  alternatives 
prospective  systems  for  the  purpose  of  identifying  the  perferred 
system  and  its  associated  equipment,  organizations,  etc.  The 
examination  aims  at  finding  more  precise  answers  to  a  question 
and  not  at  justifying  a  conclusion.  The  analytical  process 
includes  tradeoffs  among  alternatives,  design  of  additional 
alternatives;  and  the  measurement  of  the  effectiveness  and  cost 
of  the  alternatives. 

Critical  issues.  Those  aspects  of  a  system's  capability,  either  operational, 
technical,  or  other,  that  must  be  questioned  before  a  system's 
overall  suitability  can  be  known,  and  which  are  of  primary  importance 
to  the  decision  authority  in  reaching  a  decision  to  allow  the  system 
to  advance  into  the  next  phase  of  development. 

Decision  Coordinating  Paper  (DCF).  The  principal  document  to  record 

essential  system  program  information  for  use  in  support  of  the  Secre¬ 
tary  of  Defense  decision-making  process  at  Milestones  I,  II,  and  III. 

Defense  Acquisition  Executive  (DAE).  The  principal  advisor  and  staff 

assistant  to  the  Secretary  of  Defense  and  the  focal  point  in  the 
Office  of  the  Secretary  of  Defense  (OSD)  for  systems  acquisition. 

Defense  Mission.  The  mission  of  the  Department  of  Defense  (DOD)  as  specified 
by  the  legislative  authority. 

Defense  Systems  Acquisition  Review  Council  (DSARC).  An  advisory  body  to 
the  Secretary  of  Defense  on  major  systems  acquisition.  The 
Council  members  are  the  OSD  staff  principals. 

Demonstration  and  validation  phase .  The  period  of  the  acquisition  process 

when  selected  candidate  solutions  are  refined  through  extensive  study 
and  analyses;  hardware  development,  if  appropriate;  tests;  and  evalua¬ 
tions.  The  objective  is  to  validate  one  or  more  of  the  selected 
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solutions  and  give  a  basis  for  deciding  whether  to  proceed  into 
full-scale  engineering  development. 

Deployment  phase.  The  period  of  the  acquisition  process  encompassing 
the  process  of  uniting  facilities,  hardware  and  software, 
personnel  and  procedural  publications;  and  delivering  an 
acceptable  integrated  system  to  the  using  and  supporting 
commands.  This  overlaps  the  production  phase. 

Design  to  cost  (DTC).  Management  concept  wherein  rigorous  cost  goals 
are  established  during  development  and  the  control  of  systems 
costs  (acquisition,  operating,  and  support)  to  these  goals  is 
achieved  by  practical  tradeoffs  between  operational  capability, 
performance,  cost,  and  schedule.  Cost,  as  'a  key  design 
parameter,  is  addressed  on  a  continuing  basis  and  as  an 
inherent  part  of  the  development  and  production  process. 

Development  Test  I  (DT  I).  This  test  is  conducted  early  in  the  develop¬ 
ment  cycle,  normally  during  the  validation  phase.  Components, 
subsystems,  or  the  entire  system  are  examined  to  determine 
whether  the  system  is  ready  for  full-scale  development.  State- 
of-the-art  technology  is  addressed  in  DT  I. 

Development  Test  II  (DT  II).  This  test  provides  the  technical  data 
necessary  to  assess  whether  the  system  is  ready  for  low  rate 
initial  or  full-scale  production.  It  measures  the  technical 
performance  and  safety  characteristics  of  the  item  and  evaluates 
its  associated  tools,  test  equipment,  training  package,  and 
maintenance  test  package  as  described  in  the  development  plan. 

DT  II  addresses  accomplishment  of  engineer  design  goals. 

Development  test  and  evaluation  (DT&E).  That  test  and  evaluation  con¬ 
ducted  to  assist  in  engineering  design  and  development  process 
and  verify  attainment  of  technical  performance  specifications 
and  objectives. 

Engineering  change  proposal,  a  proposal  to  the  responsible  authority 
recommending  that  a  change  to  an  original  item  of  equipment  be 
considered;  and  the  design  or  engineering  change  be  incorporated 
into  the  article  to  modify,  add  to,  delete  or  supersede  original 
parts. 

Five  year  defense  program  (FYDP) .  The  basic  DOD  programming  document 

for  the  entire  military  establishment;  an  integrated,  coordinated 
program  of  forces,  costs,  manpower,  procurement,  and  construction 
projected  over  a  four-year  period  beyond  the  budget  year. 

The  FYDP  consists  of  summary  tables  supported  by  detailed 
submissions  of  the  military  departments  and  other  DOD  components. 
The  FYDP  is  formally  approved  by  the  Secretary  of  Defense  and 
is  binding,  for  programming  purposes,  on  all  components  of  the 
DOD. 
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Full  scale  development  phase.  The  period  of  the  acquisition  process 
when  the  system  and  the  principal  items  necessary  for  its 
support  are  designed,  fabricated,  tested,  and  evaluated.  The 
intended  output  is,  as  a  minimum:  a  preproduction  system  that 
closely  approximates  the  final  product;  the  documentation  needed 
to  enter  the  production  phase;  and  the  test  results  that  show 
the  product  will  meet  the  requirements.  This  phase  includes 
the  procurement  of  long  lead  production  items  and  limited  pro¬ 
duction  for  OT&E . 

Independent  cost  estimate,  a n  estimate  of  program  cost  developed  outside 
normal  advocacy  channels  by  a  team  which  generally  includes 
representation  from  Cost  Analysis  Procurement,  Production 
Management,  Engineering  and  Program  Management. 

Initial  operational  test  and  evaluation  (IOT&E).  That  portion  of 
operational  test  and  evaluation  conducted  prior  to  the 
Milestone  III  decision. 

Interoperability.  The  ability  of  systems,  units,  or  forces  to  provide 
services  to,  and  accept  services  from,  other  systems, 
units  or  forces,  and  to  use  the  services  so  exchanged  to  enable 
them  to  operate  effectively  together. 

Life  cycle  cost  ( LCC ).  The  total  cost  to  the  Government  of  a  system 
over  its  full  life.  It  includes  the  cost  of  development, 
acquisition,  operation,  support,  and  where  applicable,  disposal. 

Line  authority .  DOD  officials  in  the  direct  chain  of  authority  from 
the  Secretary  of  Defense  to  the  program  manager  and  excluding 
staffs. 

Logistics  supportability .  The  degree  to  which  adequate  provisions  can 
be  made  in  system's  acquisition  for  support  and  test  equipment, 
supply  support,  maintenance  manuals,  technical  data,  and 
support  facilities. 

Maintainability .  a  characteristic  of  design  and  installation  which 

inherently  provides  for  an  item  to  be  retained  in  or  restored 
to  a  specified  condition  within  a  given  time,  when  it  is 
maintained  in  accordance  with  prescribed  procedures  and  resources. 

Major  systems  acquisition,  a  system  acquisition  program  designated  by 
the  Secretary  of  Defense  to  be  of  such  importance  and  priority 
as  to  require  special  management  attention. 

Military  operational  requirements.  The  formal  expression  of  a  military 
need,  response  to  which  results  in  development  or  acquisition 
of  items,  equipments,  or  systems. 
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Mission  area.  A  segment  of  the  defense  mission  as  established  by 
the  Secretary  of  Defense. 

Mission  element.  A  segment  of  a  mission  area  critical  to  the  accomplish¬ 
ment  of  the  mission  area  objectives  and  corresponding  to  a 
recommendation  for  a  major  system  capability  as  determined 
by  a  DOD  Component. 

Mission  element  need  statement  (MENS).  A  statement  prepared  by  a  DOD 

Component  to  identify  and  support  the  need  for  a  new  or  improved 
mission  capability.  The  mission  need  may  be  the  result  of  a 
projected  deficiency  or  obsolesence  in  existing  systems, 
a  technological  opportunity,  or  an  opportunity  to  reduce 
operating  cost.  The  MENS  is  submitted  to  the  Secretary  of 
Defense  for  a  Milestone  0  decision. 

Mission  need.  a  required  capability  which  is  within  an  agency's 

overall  purpose,  including  cost  and  schedule  considerations. 

Operational  effectiveness.  The  overall  degree  of  mission  accomplishment 
of  a  system  used  by  representative  troops  in  the  context  of  the 
organization,  doctrine,  tactics,  threat,  and  environment  in  the 
planned  operational  employment  of  the  system. 

Operational  Test  I  ( OT  I).  OT  I  is  an  operational  test  of  a  hardware 
configuration  of  a  system,  or  components  thereof  to  provide 
an  indication  of  military  utility  and  worth  to  the  user. 

Testing  should  refine  identified  critical  issues,  report  areas 
that  should  be  addressed  in  future  OT  and  identify  new  ones  for 
subsequent  testing.  OT  I  is  accomplished  during  the  Validation 
Phase  on  brassboard  configuration,  experimental  prototypes  to 
provide  data  leading  to  the  decision  to  enter  Full-scale 
Engineering  Development . 

Operational  Test  II  (OT  II).  OT  II  is  the  test  of  engineering  develop¬ 
ment  prototype  equipment  prior  to  the  initial  product  decision. 

Its  goal  is  to  estimate  an  item's  military  utility,  operational 
effectiveness,  and  operational  suitability  in  as  realistic  an 
operational  environment  as  possible.  Test  objectives  are  based 
on  the  critical  issues  which  are  best  examined  by  using  elements 
of /or  complete  TOE  troop  units  in  controlled  field  exercises. 

Operational  test  and  evaluation  (OT&E).  That  test  and  evaluation 

conducted  to  estimate  a  system's  operational  effectiveness  and 
operational  suitability,  as  well  as  the  need  for  any  modifications. 
It  is  accomplished  by  operational  and  support  personnel  of  the 
types  and  qualifications  expected  to  use  and  maintain  the  system 
when  deployed  and  is  conducted  in  as  realistic  and  operational 
environment  as  possible. 
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Planning.  The  selection  of  courses  of  action  through  a  systematic 

consideration  of  alternatives  in  order  to  attain  organizational 
objectives. 

Planning,  programming,  budgating  system  (PPBS) .  An  integrated  system 

for  the  establishment,  maintenance,  and  revision  of  the  FYDP  and 
the  DOD  budget. 

Pre-production  model.  An  article  in  final  form  employing  standard 

parts,  representative  of  articles  to  be  produced  subsequently 
in  a  production  line  and  capable  of  being  operationally  tested. 

Production  phase.  The  period  of  the  acquisition  process  from  production 
approval  until  the  last  system  is  delivered  and  accepted.  The 
objective  is  to  efficiently  produce  and  deliver  effective  and 
supportable  systems  to  the  operating  units.  This  includes 
the  production  of  all  principal  and  support  equipment. 

Program.  An  organized  set  of  activities  directed  toward  a  common 

purpose,  objective,  or  goal  which  are  undertaken  or  proposed 
by  an  agency  in  order  to  carry  out  responsibilities  assigned 
to  it. 

Program  element.  A  description  of  a  mission  by  the  identification  of 

the  organizational  entities  and  resources  needed  to  perform  the 
assigned  mission.  Resources  consist  of  forces,  manpower,  material 
quantities,  and  costs  as  applicable.  The  program  element  is  the 
basic  building  block  of  the  FYDP. 

Program  evaluation  review  technique  (PERT) .  a  technique  for  management 
of  a  program  through  to  completion  by  constructing  a  network 
model  of  integrated  activities  and  events  and  periodically 
evaluating  the  time/cost  implications  of  progress. 

Program  manager.  The  individual  in  the  DOD  chartered  to  manage  a 
major  systems  acquisition  program. 

Program  manager  charter,  a  document  approved  by  the  DOD  Component  Head 
stating  the  program  manager’s  responsibility,  authority  and 
accountability  in  the  management  of  a  major  systems  acquisition 
program . 

Programming.  The  process  of  translating  planned  military  force 

requirement  into  specific  time-phased,  scheduled  actions, 
and  of  identifying  in  relatively  precise  terms  the  resources 
required.  It  is  the  bridge  between  planning  and  budgeting. 

Program  objectives .  The  capability,  cost  and  schedule  goals  which  are 
being  sought  by  the  systems  acquisition  program  in  response 
to  a  mission  need. 


Program  objective  memorandum  (PCM),  a  memorandum  in  prescribed  format 
submitted  to  the  Secretary  of  Defense  by  the  Secretary  of  a 
military  department  of  the  Director  of  a  defense  agency  which 
recommends  the  total  resource  requirements  within  the  parameters 
of  the  published  Secretary  of  Defense  fiscal  guidance. 

Requirement.  The  need  or  demand  for  personnel,  equipment,  facilities, 
other  resources  or  services,  expressed  in  specific  quantities 
for  specific  time  periods. 

Reliability.  a  fundamental  characteristic  of  an  item  of  material 

expressed  as  the  probability  that  it  will  perform  its  intended 
function  for  a  specified  period  of  time  under  stated  conditions. 

Request  for  ’proposal  (RFP).  The  solicited  document  between  the  Government 
and  Contractor  on  a  contemplated  procurement.  It  is  the 
medium  by  which  a  contractor  is  introduced  to  the  job  desired 
by  conveying  a  complete  understanding  of  the  work  to  be 
performed  and  to  determine  the  capability  and  price  of  the 
contractors  efforts.  Other  forms  of  solicitation  documents 
include  the  invitation  for  bid  and  Request  for  Quotation. 

Risk  assessment.  The  process  of  subjectively  determining  the  probability 
that  a  specific  interplay  of  performance,  schedule,  and  cost 
as  an  objective,  will  or  will  not  be  attained  along  the  planned 
course  of  action. 

Survivability.  The  degree  to  which  a  system  is  able  to  avoid  or  withstand 
a  man-made  hostile  environment  without  suffering  an  abortive 
impairment  of  its  ability  to  accomplish  its  designated  mission. 

Systems  acquisition  process,  a  sequence  of  specified  decisions  events 
and  phases  of  activity  directed  t.o  achievement  of  established 
program  objectives  in  the  acquisition  of  Defense  systems.  This 
process  extends  from  approval  of  a  mission  need  through  success¬ 
ful  deployment  of  the  Defense  system  or  termination  of  the 
program.  The  sequence  of  these  decisions  and  phases  are  as 
follows: 


Milestone 

Decision 

Phase 

0 

Program  initiation 

Conceptual 

I 

Demonstration  and 
validation 

Demonstration  and 
validation 

II 

Full-scale  engineer¬ 
ing  development 

Full-scale  engineer¬ 
ing  development 

III 

Production/deployment 

Production  &  Deploy- 

ment 
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(Service)  systems  acquisition  review  council  ( [S]SARC) .  The  Council 
established  by  the  Head  of  a  military  department  as  an 
advisory  body  to  him  and  through  him  to  the  Secretary  of 
Defense  on  major  systems  acquisitions.  The  (S)SARC  is  chaired 
by  the  Secretary/Under  Secretary  of  the  military  department 
and  is  similar  in  functional  composition,  responsibilities  and 
operation  to  the  DSARC.  In  application  the  term  (Service)  is 
replaced  by  the  designation  of  the  applicable  military  depart¬ 
ment,  i.e.,  ASARC  (Army),  DNS ARC  (Department  of  the  Navy), 

AFSARC  (Air  Force),  and  (M)SARC  (Marine  Corps). 

Standardization  (NATO).  The  process  by  which  NATO  nations  achieve  the 

closest  practicable  cooperation  among  their  forces;  facilitate 
the  most  efficient  use  of  research,  development  and  production 
resources;  and  agree  to  adopt  on  the  broadest  possible  basis 
the  use  of  (a)  common  or  compatible  operational,  administrative, 
and  logistic  procedures,  (b)  common,  compatible  or  interchange¬ 
able  supplies,  components,  weapons  or  equipment.  (c)  common 
or  compatible  technical  procedures  and  criteria,  (d)  common 
or  compatible  tactical  doctrine  with  corresponding  organizational 
compatibility. 

System  design  concept.  An  idea  expressed  in  terms  of  general  performance, 
capabilities,  and  characteristics  of  hardware  and  software 
which  is  oriented  either  to  operate  or  to  be  operated  as 
an  integrated  whole  in  meeting  a  mission  need. 

System  program  office.  The  office  of  the  program  manager  and  single 
point  of  contact  with  industry.  Government  agencies  and  other 
activities  participating  in  the  systems  acquisition  process. 

Test  criteria.  standards  by  which  test  results  and  outcome  are  judged. 

Thresholds.  Monetary,  time,  or  resource  limitations  placed  on  a  program, 
to  be  used  as  guides  as  the  program  progresses  and  the  breaching 
of  which  is  cause  for  careful  review  of  at  least  some  aspects  of 
the  program. 

Vulnerability .  The  characteristics  of  a  system  which  causes  it  to 
suffer  a  definite  degradation  (incapability  to  perform  the 
designated  mission)  as  a  result  of  having  been  subjected  to 
a  certain  level  of  effects  in  an  unnatural  (man-made)  hostile 
environment . 
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PERSONNEL  INTERVIEWED  FOR  THE  DEPARTMENT  OF  DEFENSE 
SYSTEMS-ACQUISITION  STUDY 


Mr.  David  Anderson 

Assistant  to  the  Executive  Secretary  of  the  DSARC 
Office  of  the  Secretary  of  Defense 
Washington,  DC  20301 

Major  General  Fred  J.  Ascani,  USAF  (Retired) 

Defense  Systems  Management  College 
Fort  Belvoir,  VA  22060 

Adjunct  Professor,  University  of  Southern  .California 

Colonel  Robert  M.  Balzihiser,  USA 

Deputy  Director,  Systems  Review  and  Analysis 
Office  of  the  Deputy  Chief  of  Staff  for 
Research,  Development,  and  Acquisition 
Headquarters  Department  of  the  Army 
Washington,  DC  20310 

Mr.  Raymond  D.  Brancolini 

Program  Manager  (GS-14) 

Marine  Air/Ground  Intelligence  System  (MAGIS) 

Naval  Surface  Weapons  Center 
Dahlgren,  VA  22448 

Colonel  Daniel  C.  Daly,  USMC 
Code  RDS-40 

Headquarters  United  States  Marine  Corps 
Washington,  DC  20380 

Rear  Admiral  R.  G.  Freeman,  III,  USN 
Commandant 

Defense  Systems  Management  College 
Fort  Belvoir,  VA  22060 

Dr.  Jacques  S.  Gansler 
Vice  President 

The  Analytic  Sciences  Corporation 
Suite  1201 

1601  North  Kent  Street 
Arlington,  VA  22209 

Former  Deputy  Assistant  Secretary  of  Defense, 
Material  Acquisition 

Dr.  Donald  W.  Hurta 

Chief;  Policy,  Analysis,  and  Process  Division 
Defense  Systems  Management  College 
Fort  Belvoir,  VA  22060 
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Lieutenant  Colonel  Robert  B.  Machen,  USA 

Director,  Directorate  of  Research 
Defense  Systems  Management  College 
Fort  Belvoir,  VA  22060 

Mr.  Joseph  Masciarelli 

Special  Assistant  for  Program  Appraisal 
Office  of  the  Secretary  of  the  Navy 
Washington,  DC  20350 

Major  General  W.  B.  Maxson,  OSAF 

Director  of  Development  and  Programming 
Office  of  the  Deputy  Chief  of  Staff  for 
Research  and  Development 
Headquarters  Department  of  the  Air  Force 
Washington,  DC  20330 

Major  General  Robert  Scurlock,  USAF 
Budget  Director 
Code  ACB 

Headquarters  Department  of  the  Air  Force 
Washington,  DC  20330 

Former  Program  Manager,  F-15 

Dr.  A.  L.  Slafsosky 

Scientific  Advisor  to  the  Commandant 
of  the  Marine  Corps 
Code  RD 

Headquarters  United  States  Marine  Corps 
Washington,  DC  20380 

Colonel  James  A.  Stempson,  USAF 

Chief  Formulation  and  Analysis  Division 
Office  of  the  Deputy  Chief  of  Staff  for 
Research  and  Development 
Headquarters ,  Department  of  the  Air  Force 
Washington,  DC  20330 

Captain  Tommy  J.  Trask,  USAF 

Acting  Executive  Secretary  to  the 
Air  Force  SARC 
Office  of  Management  Policy 
Code  RDPXM 

Headquarters,  Department  of  the  Air  Force 
Washington,  DC  20330 

Commander  Ronald  C.  Trossback,  USN 

Assistant  for  R&D  Policy  and  Acquisition  Planning 
Code  OP-980C 

Office  of  the  Chief  of  Naval  Operations 
Washington,  DC  20350 


APPENDIX  C 


INTERVIEW  GUIDE  FOR 
DEFENSE  SYSTEMS  ACQUISITION 


APPENDIX  C 


INTERVIEW  GUIDE  FOR 
DEFENSE  SYSTEMS  ACQUISITION 

Name: 

Address: 

Title: 

Responsibilities: 


1.  Background  on  the  current  DOD  Systems-Acquisition  Process. 

a.  What  has  been  the  major  impact  of  Office  of  Management 
and  Budget  Circular  A-109? 

b.  How  has  the  implementation  of  DOD  directives  5000.1  and  5000.2 
changed  the  systems-acquisition  process? 

c.  What  revisions  were  required  in  your  service's  management  of 
systems  acquisition?  Procedural?  Organizational? 

d.  What  is  the  number  of  major  systems-acquisition  programs  currently 
active  in  your  service? 

e.  How  many  (S)SARCs  are  held  each  year? 

2.  Nature  of  the  (S)SARC  and  DSARC  milestones.  How  do  the  characteristics 
of  the  four  decision  milestones  differ  considering  the  development 
programs? 

a.  Consider  risk  and  uncertainty? 

b.  Consider  cost? 

c.  Considering  the  ability  to  meet  development  schedule? 

d.  Considering  interfaces  with  other  systems? 

e.  Considering  insight  into  the  ability  of  the  system  to  meet 
operational  requirements? 

f.  Considering  the  strength  of  support  for  the  system  by  various 
service  factions? 

3.  Characteristics  of  the  members  of  the  (S)SAR C  or  DSARC. 

a.  Military  vs.  civilian  backgrounds? 

b.  Experience? 

c.  Educational  backgrounds? 

1.  Technical  backgrounds  and  academic  levels? 

2.  Top-level  school  graduates? 
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d.  Are  the  positions  as  shown  in  the  directives  always  represented 
on  the  panels? 

4.  Value  system  operative  in  (S)SARC  or  DSARC. 

a.  Is  the  objective  to  increase  national  defense  capabilities? 

b.  Is  the  objective  always  to  respond  to  the  new  or  revised 
enemy  threat? 

c.  Is  the  objective  to  represent  the  interests  of  one's  own 
organization  or  staff  section? 

d.  Is  the  objective  to  insure  survival  of  the  organization  or 
one  of  its  functions  (for  example,  interservice  rivalry)? 

e.  Is  the  objective  to  push  a  particular  technology  and/or  system 
as  a  supporting  item  for  existing  or  follow-on  developments? 

f.  Is  the  objective  to  push  a  particular  alternative  because  it  is 
politically  or  financially  feasible? 

5.  Preparations  for  the  (S)SARC  and  DSARC. 

a.  How  is  material  to  be  presented  developed? 

b.  Who  does  the  analysis? 

c.  Who  presents  the  material? 

d.  Are  several  options  presented?  In  priority  order? 

e.  What  is  the  technical  level  of  the  presentations? 

f.  Are  the  proceedings  ever  stopped  and  a  request  put  out  for  further 
analysis  and  new  information? 

6.  The  decision  process  within  the  (S)SARC  and  DSARC. 

a.  Are  there  cases  where  the  decisions  are  will  on  the  way  to 
being  set  prior  to  the  meetings? 

b.  How  much  impact  do  the  actual  presentations  have  on  the  decision 
policy? 

c.  Are  options  sometimes  revised  as  a  result  of  the  discussions 
in  the  (S)SARC? 

d.  Is  there  a  voting  process? 

e.  Is  the  Chariman's  vote  the  most  important  one? 

f.  Are  alternatives  normally  put  into  priority  sequence  or  is  only 
the  "best"  alternative  shown  on  the  decision  paper  from  the 
Chairman? 

7.  The  overall  decision  process  for  systems  acquisition. 

a.  What  are  the  results  when  the  (S)SARC  positions  are  reviewed  by 

the  Service  Secretary?  Agreement?  Minor  changes?  Total  Revision? 
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b.  Is  the  material  presented  to  the  DSARC  similar  in  content  to 
that  presented  to  the  (S)SARC?  Are  analysis,  preparations, 
and  presentation  procedures  the  same? 

c.  Do  the  services  tend  to  develop  a  single  "service"  position 
prior  to  the  DSARC  then  brief  only  this  single  alternative? 

d.  What  are  the  results  when  the  DSARC  positions  are  reviewed  by 
the  Secretary  of  Defense?  Agreement?  Minor  changes?  Total 
revisions? 

8.  Identify  the  differences  between  the  policies  for  systems  acquisition 

and  actual  practice  within  DOD. 

a.  Are  there  differences  in  the  development,  processing,  and  timing 
of  documentation,  such  as,  the  DCP  and  MENS? 

b.  Are  decisions  sometimes  made  by  other  committees ,  groups,  or 
persons  external  to  the  DSARCs  and  (S)SARCs? 

c.  How  does  the  system  handle  decisions  which  may  be  driven  by 
sources  outside  DOD,  such  as,  the  President  or  Congress? 

d.  Does  the  lobbying  of  contractors  or  other  interest  groups  induce 
differences  in  the  way  a  system  is  handled? 


e.  Are  there  other  differences? 


